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FOREWORD 

As the percentage of English language learners among the adult education 
population has grown, so has the demand for quality programs teaching 
English to speakers of other languages (ESOL). Different types of programs 
have arisen to meet the diverse needs of both learners and program funding 
agencies. These types include lifeskills, family literacy, workplace literacy, 
citizenship, pre-employment, vocational, and pre-academic. A key to quality 
programs, no matter what the type, is establishing a coherent plan for imple- 
menting and evaluating instruction and assessing learner progress. Different 
stakeholders — learners, program staff, and funding agencies — may need 
different information from an evaluation. Funding agencies are interested in 
seeing that their money is well spent, program staff want feedback on how well 
they are meeting the goals of the program and the needs of the learners, and 
learners want to see how they are progressing. All of these demands may not be 
satisfied by a single approach to assessment. A combination of standardized 
and alternative assessments can provide more complete evaluation informa- 
tion. Assessing Success in Family Literacy and Adult ESL contributes valuable 
suggestions for designing a comprehensive evaluation plan that yields accurate 
and useful information for designing, modifying, and improving programs. 

Over the past decade, program evaluation has played an increasing role in 
the federal legislation that supports adult education programs. The Workforce 
Investment Act (WIA) of 1998 calls for a comprehensive performance ac- 
countability system to assess the effectiveness of programs seeking federal 
funds for adult education and literacy services. The National Reporting System 
(NRS), which was developed to facilitate the accountability and reporting 
requirements of the WIA, requires states to document learners’ progress using 
a uniform, standardized assessment procedure. It is up to the individual states, 
with U.S. Department of Education approval, to determine what that procedure 
will be. 

It is my hope that this volume will assist adult ESOL programs, whether 
receiving federal funds or not, in designing an evaluation plan that satisfies the 
needs and goals of learners, program staff and funding agencies. 

Ronald S . Pugs ley, Director 
Office of Vocational and Adult Education 
Division of Adult Education and Literacy 
U.S. Department of Education 
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CHAPTER I 

APPROACHES 

TO 

ASSESSMENT 
AND EVALUATION 




Daniel D, Holt 

California Department of Education, Sacramento, California 
Carol Van Duzer 

Center for Applied Linguistics. Washington DC 

Among l he many responsibilities of staff members in instructional 
programs* for adult English language learners are the tasks of assessing 
learner progress and evaluating their programs. Having a comprehensive 
evaluation plan is key to effective program design. For programs that 
receive federal funds, effective evaluation is imperative. The Adult Educa- 
tion and Family Literacy Act (Title II of the Workforce Investment Act 
! WIA) of 1998) calls for states to award adult education monies to programs 
based on 1 2 criteria, including the degree to which a program establishes 
learner performance measures , the program’s effectiveness in meeting or 
exceeding the levels of performance called for by the measures, and the 
maintenance of a high-quality information management system to report 
learner outcomes and monitor program performance against the measures 
(WIA. section 23 l.c). 

The purpose of this handbook is to help staff members develop an effective 
evaluation plan for programs that serve adult English learners. It is a revi- 
sion of a handbook that focused on evaluating/fl/w/Vy literacy projects (Holt, 
1994). By integrating alternative assessment approaches with standardized 



* Sec ihe Glossary on pact's 1 2<). 131 of this publication for definitions of italicized key terms that 
appear in this chapter. 








An effective programs 
evaluation plan provides 
learners and staff members 
with accurate and useful 
information for designing, 
modifying, and improving their 
programs. 



assessment measures, staff can obtain more accurate and complete 
information to improve their programs. Much of the information 
contained in the previous edition of this handbook was based on the 
experiences of staff members and learners in projects funded under the 
Elementary and Secondary Education Act (ESEA) 'Title VII Family 
English Literacy Programs (FELP). This information remains timely 
and valuable, not only for family literacy programs but also for all 
programs teaching adult English language learners, regardless of the 
context — workplace, lifcskills, pre-employment, pre-academic, 
citizenship, or general language development. 

Chapter I has been revised from the 1994 version I o move beyond 
family literacy programs to encompass all programs for adults learning 
English. It begins with a discussion of the role of evaluation in English 
language programs. Next, the distinctions between standardized testing 
and alternative assessment are discussed. Then, diverse features of 
programs that need to be considered in designing evaluation plans are 
highlighted. Finally, the chapter provides an overview of Chapters II 
through V. 

As you read Chapters II through V, keep in mir.d that the refer- 
ences to family literacy programs can extend to any program that 
teaches adults learning the English language. A section on learner self- 
assessment has been added to Chapter IV. Additional resources that 
have appeared since the first edition of the handbook are listed at the 
end of Chapter V. 



Role of Evaluation 

An effective program evaluation plan provides learners and staff 
members with accurate and useful information for designing, modify- 
ing, and improving their programs. In order to create effective 
evaluation procedures, stafi and learners need to ( I ) clarify the goals 
and objectives of the program, (2) develop performance measures and 
indicators of progress in attaining these goals and objectives, and (3) 
identify the information they need to collect to determine the degree to 
which success has been achieved. 

The focus of the evaluation should be on the goals and objectives 
that learners and staff members have established. For example, in a 
family literacy program, if the learners are composing stories they will 
read to their children, the evaluation plan should include the proce- 
dures for collecting information about the learners' compositions and 
the strategies they used for sharing them w ith their children. Or, in a 
lifeskills-based class, if learners are learning how to negotiate for 
medical care for family members, evaluation might include procedures 
for collecting evidence that learners are able to schedule appointments, 
understand insurance benefits, and advocate fora specific treatment 
protocol. 
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Objective* should be 
clear statements representing 
a range of possible outcomes 
that are refined as the program 
is implemented. 





This volume contains suggestions on how objectives can 
be used to help program staff define the outcomes or changes 
that are expected as a result of the Instructional program. The 
content of objectives is determined by what the learners need, 
what the staff members are prepared to teach, and what is 
possible to achieve within the time and budget constraints of 
the program. 

During the initial planning of a program, staff members 
can use the development of objectives as a way to focus the 
direction of instruction based on the results of the needs 
assessment. For example, will the program concentrate on 
listening, speaking, reading, or writing skills— or some combi- 
nation? To what extent wilt the learners’ native language 
abilities be developed? How will the program incorporate the 
needs and wants of the learners (e.g., help parents support 
their children’s school achievement, enable learners to 
access community resources, provide skills to get or advance 
in a job)? 

Some objectives may deal with clearly observable out- 
comes such as the ability to read or write a piece of text; 
others may specify changes that are more difficult to see such 
as self-confidence, group participation, or literacy awareness. 
Each objective should specify only one outcome. Whatever 
the outcome, an objective should include assessment proce- 
dures and instruments that can be used to measure the extent 
to which the objective Is being achieved. 

Objectives should not be immutable, rigid directives fiom 
the staff to the learners. Nor should they be developed by staff 
without involving the learners or assessing their needs. 

Rather, objectives should be clear statements representing a 
range of possible outcomes that are refined as the program is 
implemented. Among the myriad literacy-related activities that 
a program might address, objectives specify where the staff 
and learners could begin their collaborative effort toward 
building a successful program. 



Assessment Versus Evaluation 

In this handbook, assessment refers to the use of instruments and 
procedures to gather data on a regular basis. Assessment may focus on 
identifying learners’ needs, documenting learners’ progress toward 
meeting their own goals, and ascertaining the extent to which the 
program objectives are being met. 

Evaluation refers to the integration and analysis of assessment 
data at given points for such puq) 0 ses as ( 1 ) interpreting learners’ 
needs, (2) developing goals and objectives, (3) designing the content 
of the curriculum, (4) selecting instructional approaches, (5) monitor- 
ing the implementation of the program, (6) identifying obstacles to 
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Evaluation is an 
ongoing, collaborative effort 
by learners, staff members, 
and evaluators to clarify 
learners’ needs, refine 
program goals, design 
curriculum content, and 
develop instructional 
methodology. 



achieving objectives, and (7) determining the overall success of the 
program. Examples of evaluation activities include the information 
management system reports required by some states to meet federal 
funding requirements and the annual evaluation report that is required 
of programs funded by the Office of Bilingual Education and Minority 
Languages Affairs (OBEMLA), U.S. Department of Education, 

This handbook presents evaluation as part of a comprehensive 
process of planning, implementing, and improving a program (see 
Chapter II for a description of the planning process). Evaluation should 
not be an isolated event that is conducted at the end of each year. Rather, 
evaluation is an ongoing, collaborative effort by learners, staff mem- 
bers, and evaluators to clarify learners’ needs, refine program goals, 
design curriculum content, and develop instructional methodology. The 
essence of the evaluation is the participants' specification of what 
constitutes success in their program, what approaches they will use to 
assess the level of success they have achieved, and how evaluation 
results will be used to improve the program. 

Standardized and Alternative Assessments 

This handbook is based on the assumption that assessment and 
evaluation are carried out using a variety of approaches. Some ap- 
proaches involve the use of standardized tests, such as the Basic English 
Skills Test ( BEST) or the Comprehensive Adult Student Assessment 
System (CASAS). In contrast, alternative approaches refer to proce- 
dures and instruments that are tailored specifically to ihe learners, 
curriculum, and overall design of the program. In this section, standard- 
ized tests and alternative assessment approaches are described. 

Standardized Tests 

Standardized tests are created according to explicit specifications, 
with test items selected for difficulty and discriminating power. Admin- 
istration and scoring procedures are uniform and consistent. There are 
generally two types of standardized tests — norm-referenced and crite- 
rion-referenced. Norm-referenced tests allow for comparisons between 
individuals' current achievement and the average performance (norms) 
of selected participants (nooning group). Criterion-referenced tests 
compare everyone's scores to an absolute standard (criterion of perfor- 
mance). 

Strengths of standardized tests . Standardized tests offer many 
advantages to literacy program staff. The tests can be obtained readily, 
administered easily, and scored immediately and accurately in a cost- 
effective manner. Their validity and reliability can be documented; 
many studies have been done by or for test publishers that support the 
technical merits of the tests. Staff can use standardized test results to 
compare the performance of program participants with the norms or the 
criteria established by the test developers. 
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Limitations of standardized tests. Standardized tests, 
however, provide only part of the information needed to docu- 
ment learners’ progress. For example, the results of standardized 
tests may not be useful or meaningful to staff members if the 
content of the tests is not related to the goals and curriculum of 
the program. The tests may measure discrete skills such as word 
recognition, but the program may be designed to strengthen 
shared literacy activities between parents and children or to 
develop workplace skills. In addition, standardized tests may be 
based on a norming group that differs from the participants in the 
program, thereby making it difficult to compare the participants’ 
performance with the standard. 

Because standardized tests usually focus on products (i.e., 
responses to specific items), they may fail to provide useful 
information about important presses such as enjoying books, 
obtaining needed medical services, or receiving a job promotion. 
Furthermore, many teachers may not fully understand how to 
interpret and use the test scores. Even when staff are fully trained 
in the use of standardized tests, the scores may not provide 
enough information about stude.it learning that is useful for 
making decisions about the next steps that should be taken in 
designing activities, selecting instructional materials, or refining 
program goals. 

Appropriate uses of standardized tests . The following 
guidelines should be kept in mind when considering the use of 
standardized tests: 

1 . Select a test that measures at least 40% of the content taught 
in the program. 

2. Select a test that is sensitive to the cultural, linguistic, and 
age differences of the learners. 

3. Interpret test scores carefully, because no test is perfect, and 
each score is only an estimate of the learner’s true ability. 

4. Never use one cut-off score. Create a range or bandwidth of 
scores within which further assessment is carried out. For 
example, a score below 40% correct indicates that the 
student needs further learning experiences; 40% to 50%, 
more assessments are necessary to judge the learner’s 
achievement; and over 50%, the learner can move to a new 
learning area. 

5. Examine carefully the background information about the 
test to determine the test’s relevance for the program (for 
example, the composition of the norming groups and the 
contents of the test). 



13 



Chapter i 




To find a standardized 



test: 

• Review the objectives of the 
program. 

• Determine the viability of a 
few te*ts. 

• Use one test on a group of 
learners. 

• Provide the learners with 
test-taking skills. 



6. Compare the *est results with the learner's performance in the 
classroom to ensure that learners with higher scores on the test 
are those who perform better in class. 

7. Consider the results of supplemental assessments and other 
information about the learner so that decisions are not based on 
one standardized test score. (Evaluation Assistance Center-West, 
1992) 

Finding an appropriate standardized test. By following the 
guidelines above, program staff may be able to identify a standardized 
test that meets the needs of their program. The following steps are 
suggested for selecting an appropriate standardized test: 

1. Review the objectives of the program. If necessary, put them in 
hierarchical order with the most important objective first and the 
least important last. 

2. Identify four or five potential tests by contacting other similar 
programs, reading evaluation reports or research studies to 
determine what measures were utilized, contacting technical 
support agencies, and looking for reviews of tests in professional 
journals or magazines. 

3. Determine the viability of the tests by consulting professional test 
review sources (for example, The Thirteenth Mental Measure- 
ments Yearbook , edited by Impara and Plake, 1998, or Test 
Critiques , edited by Kcyser and Sweetland, 1994) to ensure that 
the test meets the needs of the program and has the appropriate 
characteristics of reliability and validity. The possibilities should 
thus be narrowed to two or three tests. 

4. Request the technical manual and a review copy of each of the 
possible tests. Determine whether there is a match between test 
objectives and program objectives and between the demograph- 
ics of the norming group for the test and the learners in the 
program. One test should now stand out as the best option. 

5. Use the test on a group of learners — ideally, learners from two 
very different skill levels — to ensure that they feel comfortable 
with the test, that the test can differentiate among learners with 
different skill levels, and that the scores provide useful informa- 
tion about the learners. 

6. If a decision is made to use the test on a regular basis, provide the 
learners with some training in test-taking skills. This does not 
mean teaching to the test but rather helping learners develop such 
strategies as making good choices among alternative responses, 
using time wisely, and skipping difficult items. 

Some of the standardized tests that are typically used with adult 
English language learners are listed in Table 1. Several books and 
articles are available that provide more information about standardized 
tests (see Jackson, 1990; Sticht, 1990; Van Duzer & Berdan, 1999). 
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Standardized Tests Commonly Used in Programs for 
Adults Learning English 



Test 


Purpose 


Basic English Skills Test 
(BEST) 


Listening, speaking, reading, 
writing 


Comprehensive Adult 
Student Assessment System 
(CASAS) 


Lifeskills, listening, reading 


Aduit Language Assessment 
Scales (A-LAS) 


Listening, speaking, reading, 
writing, math 


Spanish Assessment of 
Basic Skills (SABE) 


Reading, math 



I Alternative approaches 
allow staff to identify what is 
important in their program and 
select assessment strategies 
that are tailored to the unique 
characteristics of the learners. 



It is generally recognized that tests developed for native English 
speakers are not appropriate for use with English language learners, 
although the Tests of Adult Basic Education (TABE) and Adult Basic 
Learning Exam (ABLE) are sometimes used with learners who have a 
considerable amount of English proficiency. 

In summary, as suggested by Navarrete, Wilde, Nelson, 

Martinez, & Hargett (1990), standardized tests of basic skills and 
language proficiency can be used if the following conditions are met; 

1. They are not the sole measurement of learners’ achievements. 

2. Scores are interpreted carefully. 

3. They match the demographic and educational backgrounds of the 
learners in the program. 

4. The objectives of the test match the objectives of the program, 
including the curriculum content. 

Alternative Approaches 

Alternative approaches to assessment and evaluation may be 
characterized as flexible, representative of the curriculum, meaningful 
to learners, and indicative of learners' acquired ability or knowledge 
(Navarrete et a! M 1990). Alternative approaches allow staff to identify 
what is important in their program and select assessment strategies that 
are tailored to the unique characteristics of the learners. Unlike stan- 
dardized tests, alternative approaches can identify the learners' 
strengths and needs without comparing results with an external stan- 
dard or norming group. Alternative approaches provide 
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Surveys are used to 
obtain general information from 
largo nurnf are of individuals. 



Interviews are designed 
to collect detailed information. 



Observation measures arc 
used for collecting and recording 
information about various 
aspects of the program. 



Performance samples are 
examples of learners' work in 
selected tasks. 



multidimensional, highly current views of the learners 1 progress in 
many different contexts. Because the contents of the instruments can 
be linked directly to the curriculum, alternative approaches give snff 
members immediate access to feedback for planning subsequent 
learning activities. 

Four alternative approaches are discussed in Chapters III 
through V: surveys , interviews , observation measures , and perfor- 
mance samples. Other examples of assessment alternatives include 
portfolios, journals, and investigations (described in Wrigley, 1992). 

Surveys are used to obtain general information from large 
numbers of individuals. Questions used in surveys may be open- 
ended, allowing subjects to provide their own responses, or closed, 
requiring subjects to select only from the response choices provided. 
The answers may be recorded by the respondents or by the person(s) 
administering the survey. Examples of surveys are presented in 
Chapter III. 

Interviews are designed to collect detailed information. The 
interviewer asks questions orally and may make follow-up inquiries to 
clarify or amplify responses. The interviewer may document the 
results by taking notes or tape-recording responses. Although inter- 
views are usually administered individually, they also may be 
conducted with groups of learners. Chapters III and V describe how 
interviews can be used with focus groups. 

Observation measures are used for collecting and recording 
information about such aspects of the program as learner characteris- 
tics, group interactions, or literacy performance. Observation 
measures may be used for closed-ended assessments, i.e., assessments 
with restricted evaluation choices such as a numbered rating scale, or 
they may be open-ended, i.e., a narrative description evaluating 
learner performance. An example would be when a staff member 
judges how confident a learner appears when participating in a 
cooperative learning gi oup. Chapter IV provides examples of obser- 
vation measures. 

Performance samples are examples of learners’ work in se- 
lected tasks. For instance, learners may be asked to write a story, give 
an oral presentation, participate in a role play, or read a poem. Staff 
members determine how the task will be documented and how the 
learners 1 ability to carry out the task will be assessed. Stories can be 
collected in written form, role plays can be videotaped, and poetry 
readings can be audiotaped. Performance samples differ from obser- 
vation measures in that observation measures record only the 
observer’s assessment of learners’ behavior or performance. Perfor- 
mance samples include actual learner work (e.g., writing samples) and 
the results of the assessment of the work (e.g,, holistic scores on 
writing samples). Examples of performance samples can be found in 
Chapter IV. 

A variety of techniques may be used to record the information 
that is collected in any of the four alternative assessments. For ex- 
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The authors present 
oxamples of four alternative 
approaches with the 
understanding that program 
staff members will modify the 
examples or create their own 
instruments and approaches 
based on the unique 
characteristics of learners, 
the staff, and the program. 



Any assessment 
approach has limitations, hut 
by using a variety of 
instruments, staff can help 
ensure that they will obtain a 
comprehensive view of their 
project. 



ample, a checklist might be used to record responses to questions in a 
survey. A frequency count could be taken of the answers obtained in an 
interview. A rating scale might be used to record judgments in an 
observation measure. Portfolios could be used to collect and organize 
samples of learners' writing, results of surveys and individual inter- 
views, information collected through observations, and other examples 
of learners' accomplishments. What is included in the portfolio 
depends on the objectives of the program. Staff can make the contents 
highly personalized and meaningful by asking learners to help decide 
what to place in the portfolio. 

The authors of Chapters III, IV, and V present examples of these 
four alternative approaches with the understanding that program staff 
members will modify the examples or create their own instruments and 
approaches based on the unique characteristics of the learners, the 
staff, and the program. Program staff should consider many factors 
when selecting an alternative approach. For example, open-ended 
responses may give flexibility to respondents but take additional bme 
and energy for staff to analyze. Observation measures are useful for 
documenting staff s judgments but may yield different data depending 
on which staff member is doing the observing. Any assessment ap- 
proach has limitations, but by using a variety of instruments, staff can 
help ensure that they will obtain a comprehensive view of their pro- 
gram. They may find it helpful to seek the assistance of an experienced 
evaluator to design and implement alternative assessment approaches. 
An evaluator can help staff create instruments that are psychomet.ri- 
cally sound, analyze the results of the assessments, and make 
recommendations for improving the program. 

This handbook focuses on using alternative approaches to ( l ) 
identify learners' needs, (2) determine learners' ongoing progress in 
meeting program objectives, (3) ascertain the overall success of the 
program, (4) give staff immediate and relevant feedback. (5) enable 
learners to assess their own development, and (6) help meet the evalua- 
tion guidelines established by funding sources. The authors assume 
that staff members will implement alternative approaches in conjunc- 
tion with standardized measures. By designing evaluations that 
integrate alternative approaches and standardized tests, staff will have 
in-depth information to make decisions about program improvement. 

Diversity in Programs for Adult 
English Language Learners 

Programs must address the divert goals of both learners and 
program funding agencies as well as b e external circumstances (e.g.. 
work schedules of the learners, availability of classroom space) that 
affect learner participation. This section provides some examples of 
the diversity that is possible in the design of literacy programs (see also 
Holt & Holt, 1995). 
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Programs may be located 
in iarge cities or rural settings 
and may be conducted in 
schools, community centers, 
workplaces, or the participants' 
homes. Programs may offer 
sessions during the day, 
evening, or both and for short or 
long periods of time. 





ui!g? Most programs include 
activities that promate learners' 
acquisition of English. Some may 
focus on language development, 
whereas others integrate 
language and lifeskills. Some 
focus on oral communication as 
a prelude to literacy, while 
others explicitly address the 
development of reading and 
writing concurrently. 



Participants, Location, and Schedule 

Literacy programs are characterized by diversity in terms of 
participants, location, and schedule. Learners differ strikingly in terms 
of their linguistic and cultural backgrounds, the characteristics of their 
families, the demography of their communities, and their reasons for 
wanting to improve their literacy abilities. Groups served may include 
American Indian, Cambodian, Chinese, Haitian, Hispanic, Hmong, 
Korean, Russian, Samoan, and Vietnamese. Sonne programs serve a 
homogeneous group of learners; others serve learners with diverse 
language and cultural backgrounds. The ages of participants also vary. 
Some programs focus on young adults who are not in school. Others 
focus on preparing adults to find work. Some family literacy programs 
include many opportunities for parents and children to learn together 
in intergenerational activities; others focus mainly on literacy training 
for parents. Programs may be located in large cities or rural settings 
and may be conducted in schools, community centers workplaces, or 
the participants' homes. Programs may offer sessions during the day, 
evening, or both and for short or long periods of time. 

Program Design 

Programs for adults learning English differ in the ways they 
respond to learners 1 needs and to the requirements of their sources of 
funding. Most programs include activities that promote learners' 
acquisition of English. Some may focus on language development, 
whereas others integrate language and lifeskills. Some focus on ora! 
communication as a prelude to literacy, while others explicitly address 
the development of reading and writing concurrently. Some programs 
include activities to help learners develop their abilities in English and 
their native languages. Other programs assist participants to improve 
their knowledge of topics such as mathematics, nutrition, the U.S. 
educational system, and social services. Subject matter may be taught 
in English or in a combination of English and the learners' native 
language. 

Family literacy and workplace literacy programs may be linked 
to schools or work sites outside the program. For example, some 
family literacy programs attempt to create a close link between the 
parents' learning activities and the educational experiences of their 
children. A program might be designed to help parents learn skills that 
would enable them to serve as volunteers at school or to work with 
their children at home in school-related activities. Pre-employment 
programs might focus on the language and skills needed to enter the 
workforce. Workplace literacy programs might focus on curriculum 
designed to meet the needs of the sponsoring agency (the workplace) 
as well as those of the workers. 
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Characteristics of Programs for Adult 
English Language Learners 

The special characteristics of programs for adult English language 
learners necessitate the use of assessment and evaluation approaches 
that are specifically tailored to the needs of learners and staff. This 
section presents aspects of programs that need to be considered when 
designing the evaluation. 



The special charac- 
teristics of programs for adult 
English language learners 
necessitate the use of 
assessment and evaluation 
approaches that are specifically 
tailored to the needs of 
learners and staff. 



Enrollment and Attendance 

Unlike students in elementary and secondary schools, learners in 
most adult programs enroll voluntarily. However, before they can enroll 
and benefit from services, they must know that a program exists. Staff 
members must be prepared to collect baseline data whenever new 
participants enroll. After enrollment many adult learners may have 
difficulty attending program activities because of scheduling conflicts 
caused by personal, family, or job-related issues. 

Assessment and evaluation strategies need to take into account the 
uneven enrollment and attendance patterns that characterize many 
programs. Ongoing use of alternative approaches is more effective than 
infrequent standardized tests for enabling staff to identify obstacles to 
the learners’ participation and make adjustments that will facilitate 
enrollment and attendance. 



Learners come to the 
literacy program with different 
goals and needs for acquiring 



language and literacy. 



Multiple Contexts, Goals, and Needs 

Adult learners face demands in the home, community, school, and 
workplace. Depending on the requirements of these contexts, learners 
come to the program with different goals and needs for acquiring 
language and literacy. Some adults may want to learn English to work 
in a restaurant. Others may be interested in opening a business. Some 
may need to learn English for a job interview, and others may want to 
help their children write folk tales in their native language. Still others 
may have more general goals such as learning English to communicate 
and negotiate in daily life. 

Adult learners also have different levels of knowledge about the 
school, community, and workplace. Staff members need assessments to 
obtain accurate information about learners’ knowledge and needs in 
order to plan appropriate instruction. Alternative approaches, which can 
be tailored specifically to the learners’ language and cultural back- 
grounds, provide staff with highly specialized and useful information. 
Staff members can use the assessment results for designing objectives 
and selecting instructional content and methodology that will validate 
the learners’ prior experiences and facilitate their success in the pro- 
gram. 

Use of Native Language and English Literacies 

Individuals differ according to the purposes for which they use 
their native language and English and the degree to which they depend 
on each language to carry out daily tasks. For example, in some fami- 
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Program staff need to 
know about the cultural! and 
linguistic backgrounds of adult 
learners to design and 
implement effective strategies 
for recruiting participants, 
identifying learners’ needs, and 
designing appropriate 
instructional activities. Staff 
members may need training in 
second language acquisition, 
bilingual development, and 
adult education so they can 
create effective activities for 
the learners in their program. 



By combining 
alternative and standardized 
measures, staff members and 
learners will improve the 
usefulness and accuracy of 
information they have 
available for making decisions 
about their program. 



lies, spouses may use their native language with each other but use a 
mixture of English and rlieir native language with their children. 
Learners also have different levels of proficiency and educational 
experiences in their native language. Some learners may have received 
secondary or post-secondary certification, but others may have come 
from countries in which no formal schooling was available. 

According to Quintero and Huerta-Macfas (1990), to provide 
appropriate instruction, staff need to know the learners' proficiency 
levels in and uses of both their native language and English. The lack 
of standardized instruments to assess proficiency in languages other 
than English makes the use of alternative approaches essential for 
determining the participants’ needs, competencies, and growth in their 
native languages. Chapters III and V contain suggestions for assessing 
native language proficiency. 

Staff Knowledge 

Program staff need to know about the cultural and linguistic 
backgrounds of adult learners to design and implement effective 
strategies for recruiting participants, identifying learners’ needs, and 
designing appropriate instructional activities. Staff members may need 
training in second language acquisition, bilingual development, and 
adult education so they can create effective activities for the learners in 
their program. 

When alternative approaches to assessment and evaluation are 
developed in close consultation with staff and learners, they are 
effective for helping staff improve their knowledge of learners and 
assessing the ongoing progress of the program. Chapter II emphasizes 
the importance of the development of a literacy framework — staff 
members’ and learners’ shared understanding of the learners’ language 
and literacy needs, program goals, curriculum and methodology, and 
approaches for assessing and evaluating learning activities. 

Issues to Be Considered in Evaluating Programs 

Programs funded with federal monies are guided by the evalua- 
tion requirements set forth in federal statutes and regulations. Other 
funding sources (e.g., private foundations, community-based organiza- 
tions) have their own evaluation requirements. Staff members need to 
determine how to construct an evaluation design that meets the re- 
quirements of their funding sources and provides useful information 
for assessing learners and evaluating the program. Dy combining 
alternative and standardized measures, staff members and learners will 
improve the usefulness and accuracy of information they have avail- 
able for making decisions about their program. 

In general, the requirements of most funding sources specify that 
three areas be considered in the evaluation: ( 1 ) learner outcome data 
relative to performance measures, (2) technical standards, and (3) 
implementation data. Outcome data refer to what the students learn 
during the program. Program staff need to examine the regulations and 
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determine how the required data can be obtained for all of the partici- 
pants in the program. Chapters II through V of this handbook 
emphasize the importance of collecting and analyzing outcome data 
based on the spec. . features of the program. Issues regarding the 
evaluation of program implementation that are unrelated to learner 
out* mes (for example, staff development) are not addressed in this 
volume (see, however, Program Standards for Adult Education ESOL 
Programs , TESOL, in press). 

This section presents some of the issues to be considered in the 
evaluation of programs for adult English learners. 

Comparison Group 

At least part of the evaluation design may include the assessment 
of the educational progress of program participants against an appro- 
priate non-program comparison group. Staff members can meet this 
requirement with a standardized test that compares the participants’ 
progress with the test’s norming group. To complement the results of 
the standardized test, staff could use alternative approaches to assess 
the learners’ progress without comparing their performance with a 
non-program comparison group. 

For example, current participants’ performance can be compared 
with (1) the performance of individuals who are not participating in the 
program and have similar characteristics to the participants or (2) 
baseline data that has been collected at an earlier point in the program 
from the current participants or from all participants who were en- 
rolled at an earlier time. Staff should note that they may not need to 
compare the results of every assessment with a non-program compari- 
son group. Chapter V presents suggestions for using alternative 
assessments with non-program comparison groups. 




Staff should obtain 



information about the learners’ 



performance in the major 
components of the program 
such as language development, 
literacy, parenting, and cultural 
adaptation. 



Representativeness of the Findings 

Evaluation findings should apply to the participants, schools, or 
agencies served by the program. In other words, the conclusions made 
in an evaluation report should be derived from data on learners served 
by the program and on the full range of services provided by the 
program. Although absenteeism and transiency may prevent staff from 
obtaining assessment data on all learners, data should be collected 
from a sample of learners that is representative of the learners served. 
Similarly, staff should obtain information about the learners' perfor- 
mance in the major components of the program such as language 
development, literacy, parenting, and cultural adaptation. Alternative 
assessments enable staff to tailor assessment and evaluation to the 
characteristics of the learners and the services of the program, thereby 
helping staff to ensure that the evaluation findings are indicative of the 
literacy practices that are in place. 
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A variety of alternative 
approaches should be used for 
conducting intake and initial 
assessments, monitoring 
progress, and assessing the 
overall success of the 
program. 



Validity and Reliability 

Assessment instruments and procedures, whether alternative or 
standardized, should be valid; that is, they should measure what they 
claim to measure. Assessments also should be reliable; they should 
produce similar results consistently. Test manuals for administering 
standardized tests often contain information on the reliability of the 
tests. When developing alternative assessments, staff members need to 
devote time and resources to field testing these assessments to docu- 
ment their validity and reliability (see Guidelines for Developing 
Reliable and Valid Alternative Assessmefits, Evaluation Assistance 
Center-West, 1991). Chapter V addresses issues of validity and 
reliability in evaluation design. 

Multiple Measures 

A variety of approaches should be used for conducting intake and 
initial assessments, monitoring progress, and assessing the overall 
success of the program. Multiple instruments and procedures can 
provide staff with a comprehensive view of the learners, the progress 
they are making, and the effectiveness of the activities of the program. 

Overview of the Handbook 

This handbook is intended to assist staff members in programs 
for adults learning English to design and implement alternative 
approaches to assessing and evaluating their program. Each chapter 
addresses a different aspect of assessment and evaluation. 

Chapter II describes a model for integrating program planning, 
implementation, and evaluation activities. A central element of a 
successful program is a conceptual framework that is shared among 
the staff and learners. The framework represents the staffs collective 
thinking regarding the definition of literacy; appropriate instructional 
approaches; the respective roles of staff and learners; and other factors 
that influence the design, implementation, and evaluation of the 
program. The author points out how program planning and evaluation 
processes operate in synchrony when all participants share a common 
understanding of the design and focus of the program. 

Chapter III addresses initial assessment, including needs assess- 
ment, intake, and placement. The chapter contains examples of 
alternative approaches that have been used successfully for initial 
assessment: surveys, interviews, and writing samples. The author 
suggests strategies for using initial assessments to yield accurate 
baseline information about learners so that staff can design appropriate 
learning activities and determine the extent to which the learners are 
making progress. 

Chapter IV describes how alternative assessment and evaluation 
approaches can be used to document learners’ progress toward meet- 
ing instructional objectives. Three alternative approaches are 
presented in detail: observation measures, interviews, and writing 
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samples. Using can-do lists for learner self-assessment is discussed. 
The author emphasizes the importance of ongoing assessment for ( 1 ) 
helping staff members determine the extent to which their instructional 
efforts are leading to positive results and (2) showing learners that they 
are making progress toward meeting their own goals. 

Chapter V provides suggestions for using surveys, performance 
samples, focus groups, and observation measures to demonstrate 
progress toward achieving the goals and objectives of the program. 

The author presents strategies for collecting, analyzing, and reporting 
data obtained with each of these alternative approaches. The chapter 
underscores the importance of staff members 1 and learners’ use of the 
data for (1) making decisions that will refine the goals, objectives, 
curriculum^and overall design of the program and (2) determining the 
progress made toward attaining the goals of the program. 

Summary 

This handbook is designed to help staff members design a 
comprehensive evaluation plan for programs for adult English lan- 
guage learners. No single approach to assessment can provide a 
comprehensive view of what is happening in a program. By integrating 
alternative assessment approaches with standardized assessment 
measures, staff can obtain accurate and complete information for 
evaluating their program. Four alternative approaches are discussed: 
surveys, interviews, observation measures, and performance samples. 
Each chapter presents strategies for designing and implementing 
alternative approaches that are tailored to the needs of individual 
programs. The chapters follow a sequence that can be used by staff in 
addressing assessment and evaluation issues: planning the program and 
creating the evaluation design (Chapter II); determining learners’ 
needs and establishing baseline data (Chapter III); assessing learners’ 
progress toward meeting program objectives (Chapter IV); and collect- 
ing, analyzing, using, and reporting data obtained with alternative 
approaches (Chapter V). 



By integrating 
alternative approaches with 
standardized measures, staff 
can obtain accurate and 
complete information for 
evaluating their program. 
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“If you want something done right, do it yourself’ best expresses the theme of 
this chapter. Much has been written regarding the shortcomings and failures of 
traditional assessment* and evaluation to meet the needs of adult literacy 
projects. Current research and practice in adult literacy suggest ways of 
improving the utility and relevance of assessment and research efforts. How- 
ever, the most effective alternatives to traditional approaches are created by 
staff members and adult learners as they collaborate to improve their own 
literacy projects. 

The purpose of this chapter is to describe a process for planning, implementing, 
and evaluating literacy programs for limited-English-proficient (LEP) adults, 
parents, and children. The process emphasizes the importance of using alterna- 
tive approaches for designing and evaluating literacy programs. 

The chapter is divided into four sections: ( 1) an explanation of how assessment 
and evaluation procedures are determined in part by how learning is defined; 
(2) a discussion of the requirements for an alternative assessment and evalua- 
tion design; (3) a description of the importance of a well-defined literacy 
framework for the project; and (4) a presentation of a planning process that 
project staff can use as a guide to develop the instructional program and an 
appropriate assessment and evaluation design. 



* See the Glossary on pages 129-131 of this publication for definitions of italicized key terms that appear 
in this chapter. 





The ranking of students 
results in various negative 
consequences, not the least of 
which Is that those who rank at 
or near the bottom are ignored 
In the learning process or 
otherwise stigmatized. 



Views of Learning 

How learning is defined helps determine how literacy is defined 
and evaluated. The way staff members define learning influences vvhai 
assessment information they collect, how they collect it, and how they 
use it in the project. Currently there are two views of learning with 
decidedly different implications for data collection and use. The first is 
a traditional view of learning that has dominated education since the 
turn of the century. The second view, bolstered by research in cogni- 
tive psychology, is helping to redefine evaluation. 

Traditional View of Learning 

Wolf, Bixby, Glenn, and Gardner (1991 ) explain how the field of 
evaluation has come to rely almost exclusively on standardized tests. 
Since the (urn of the century, intelligence has been viewed as a fixed 
attribute (unchangeable, like skin color), able to be ranked (somewhere 
on n normal curve), and genetically determined. Educators who hold 
this view of intelligence classify students into specific groups based on 
the assumption that they have different abilities and therefore need a 
special curriculum. Such approaches can lead educators to assign 
students permanently to groups according to their perceived abilities. 
Because intelligence is seen as a fixed attribute, learning is viewed as 
oiic dimensional, with individuals progressing through a series of 
discrete steps from simple entry level skills to full mastery. The 
perception of learning as molecular and hierarchical has guided the 
development and use of standardized tests. 

According to Wolf and others, “Standardized tests . . . are con- 
structed so that only a few can score high; they have yielded artifacts 
rather than authentic samples, promoted an isolated view of learning, 
and treated assessment as a matter of pure measurement" ( 1991, p. 47). 
The tests contribute to the view that assessment results are end prod- 
ucts and that cognition is an individual, isolated activity. Test develop- 
ers have often created instruments that are more effective at ranking 
students than identifying where they arc developmental^. The ranking 
of students results in various negative consequences, not the least of 
which is that those who rank at or near the bottom are ignored in the 
learning process or otherwise stigmatized. Contemporary research in 
developmental and cognitive psychology, however, has questioned 
traditional assumptions about learning and suggests decidedly differ- 
ent directions for data collection. 



Learning Redefined 

Current research suggests that intelligence is not a fixed phenom- 
enon but a dynamic, global capacity. All human activity requires that 
"the learner actively develop knowledge by observing, inferring, 
generating rules, and building theories’’ (Wolf et ah, 1991, p. 48). 
Learning should be viewed not as the development of a series of 
discrete skills, hut as a process involving “qualitative and uneven shifts 
in understanding" (Wolf et ah, p. 50). Learning represents a scries of 
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Assessment, 
evaluation, and other 
components of a literacy 
project need to work together 
in a mutually supportive 
manner. 



ever-increasing, complex understandings as an individual 
observes, theorizes, tests, observes further, and readjusts the 
theory. 

Learning is not only the acquisition of discrete informa- 
tion; it is also the organization of information as a basis for 
further learning. Furthermore, learning does not occur in isola- 
tion. It is affected profoundly by external stimulation from other 
individuals, environmental pressures, or resources. This redefi- 
nition of learning suggests that staff members in family literacy 
projects need to consider carefully the kind of evaluation data 
that is collected and the methods of gathering data. Assessment 
procedures should be used to determine the leamei’s ability to 
construct and apply knowledge for a variety of purposes. 

importance of a Literacy Framework 

Assessment, evaluation, and other components of a literacy 
project need to work together in a mutually supportive manner. 
One way to coordinate the components is to create a literacy 
framework. A project’s literacy framework may be described as 
the staff members’ and learners’ shared understanding of 
learners' literacy needs, program goals, curriculum and method- 
ology, and approaches for assessing and evaluating literacy 
development. 

To design and implement alternative assessment instru- 
ments, staff need to identify outcomes and changes that consti- 
tute the learners’ success in the project. In other words, how will 
the staff and learners know that progress is being made in 
attaining the objectives of the project? The staff members’ and 
learners’ shared understanding of the nature of literacy and the 
specific outcomes that the participants are working toward 
represent key parts of the project’s literacy framework. 

As was pointed out in Chapter I, family literacy projects 
have different program designs. Some projects may help parents 
improve their individual language abilities, whereas others focus 
on shared literacy activities between parents and their children. 
Developing a literacy framework is one of the first steps the staff 
and adult learners can take to create a common -‘nderstanding of 
the role of literacy in the project and in the lei ner •’ lives. 

A Collaborative Approach to 
Planning and Evaluation 

One of the keys to linking the project’s literacy framework, 
program design, and assessment and evaluation procedures is a 
collaborative approach to planning and evaluation. This section 
presents a rationale and process for creating a literacy frame- 
work for the project and describes the steps involved in a 
collaborative approach to planning and evaluation using ex- 
amples taken from actual FELP projects in California. 



27 



Chapter II © 



Alternative 
assessments offer staff the 
advantage of being able 
measure the learners’ ability to 
use literacy in contexts that 
are relevant to their daily lives. 



Learners bring to the 
project their own goals, prior 
literacy experiences, and 
attitudes toward reading, 
writing, and learning, all of 
which interact and affect what 
and how they will Searn. 



Contexts for Literacy 

Individuals and language groups differ according lo the ways in 
which they use language and literacy nnd the settings or contexts in 
which they read and write (Lytle, BelzL Schultz, & Vannozzi, 1989). 
Thus, adult learners have different goals and needs for literacy devel- 
opment. Learners may need to be able to write notes to their children's 
teacher, interpret monthly bills and bus schedules, decipher product 
labels, help their children with school work, read a repair manual, or 
complete an application for admission to a training program. Alterna- 
tive assessments offer staff the advantage of being able to measure the 
learners’ ability to use literacy in contexts that are relevant to their 
d; ily lives. 

Literacy is best viewed as a range of practices or activities 
(Scribner, 1987). In a recent research study conducted by Wrigley and 
Guth, ESL literacy instruction was defined as “supporting language- 
minority adults with little English and little formal education in their 
efforts to understand and use English in its many forms (oral and 
written, including prose, document, and quantitative literacy), in a 
variety of contexts (family, community, school, work), so that they can 
reach their fullest potential and achieve their own goals (personal, 
professional, academic)” (1992, p.7). Chapter IV provides detailed 
suggestions for how curriculum and assessment can be matched with 
this holistic view of literacy. 

Project staff need to work closely with adult learners to design 
assessment procedures that will identify the types of reading and 
writing the learners are doing and their individual “purposes and 
aspirations” (Lytle et ah, 1989, p. 55). The procedures should help 
staff and learners examine their own views of learning, literacy, and 
how literacy is used in their daily lives. Staff and learners also can 
collaborate in designing and implementing evaluation activities 
including analyzing and interpreting evaluation data. Auerbach (1992) 
emphasizes that adult learners bring to the project their own goals, 
prior literacy experiences, and attitudes toward reading, writing, and 
learning, all of which interact and affect what and how they will learn. 
Such background information needs to be used to create meaningful 
assessment and evaluation activities in the project. 

Lytle et al. (1989) designed a framework to guide the develop- 
ment of assessments and evaluations that consider the diverse needs 
and contexts of adult learners. Their framework has four dimensions: 

1 ) literacy practices; 2) reading, writing, learning strategies, and 
interests; 3) perceptions of reading and writing, teaching and learning; 
and 4) goals. Literacy practices or activities refer to how, when, and 
why learners use literacy in their lives. After the staff and learners have 
identified the role of literacy in various contexts, assessment and 
evaluation approaches can be used to measure changes in the way 
learners use literacy as the project is implemented. 

The framework outlined by Lytle et al. (1989) may be helpful to 
FELP participants who are designing their own literacy framework. 
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Collaboration among 
staff and adult learners in 
developing a literacy 
framework enables all 
participants in the project to 
increase their understanding 
of literacy and improve their 
ability to use literacy to 
achieve their own goals. 



One of the assumptions 
of an effective literacy frame- 
work is that adult learners 
acquire abilities and 
knowledge in a spiral versus 
sequential fashion, with 
increasing levels of proficiency 
and understanding emerging 
over time. 



For example, the literacy framework should include a definition of 
literacy, an identification of the role that literacy plays in learners’ 
lives in various contexts, documentation of the learners’ goals for 
literacy development, the focus of curriculum content and activities, 
and the basic direction of the assessment and evaluation procedures. 
Collaboration among staff and adult learners in developing a literacy 
framework enables all participants in the project to increase their 
understanding of literacy and improve their ability to use literacy to 
achieve their own goals. 

The next section presents suggestions for designing a literacy 
framework that is based on the learners’ needs. The literacy framework 
and planning process contain elements that were borrowed from the 
work of Lytle, Auerbach, and others. The information is presented to 
encourage projects to begin the process of creating learner-centered, 
participatory approaches to literacy development. The reader should 
understand, however, that this chapter and others in the volume contain 
suggestions, such as developing project objectives and using ready- 
made curriculum, that some advocates of participatory literacy ap- 
proaches might consider inconsistent with authentic learner-centered 
programs. 

Developing A Literacy Framework Through Collaborative 
Planning 

The development of a literacy framework is the first step in 
designing alternative assessment and evaluation procedures. The 
literacy framework is comparable to model curriculum standards that 
educators can use as a foundation for developing programs that meet 
local needs (see California Department of Education, 1992). 

One of the assumptions of an effective literacy framework is that 
adult learners acquire abilities and knowledge in a spiral versus 
sequential fashion, with increasing levels of proficiency and under- 
standing emerging over time. As staff and learners work together to 
create the framework and related curriculum, they also can develop 
project-based assessment and evaluation procedures to document 
progress toward meeting objectives and making improvements in the 
program. The importance of the literacy framework as a foundation for 
assessment is underscored by Lytle, Marmor, and Penner: “Our 
starting point was the understanding that assessment must correspond 
to program philosophy and goals, since assessment procedures embody 
and thus convey particular concepts about literacy" (1986. p. 22). 

At a meeting of selected California FELP projects, staff members 
attempted to reach consensus on a definition of literacy. The results of 
their discussion are summarized in the sever, points listed below 
(Results of the Focus Group Meeting, 199 1 , p. 4). 

1 . Literacy is finding meaning from print in order to fulfill the learners' 
academic, psychological, political, and social goals. 

2. Literacy includes a range of activities or practices in multiple contexts 
with a variety of people. It may involve the use of more than one 
language. 
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Literacy in more than 
one language strengthens 
learners’ linguistic and 
cognitive development and 
increases their scciocultural 
knowledge, 



3. Contexts differ with respect to the complexity of literacy demands. 

4. Literacy activities include reading, writing, and learning strategies. 

5. Literacy in more than one language strengthens learners' linguistic and 
cognitive development and increases their sociocultural knowledge. 

6. The development of literacy is influenced by the language, culture, 
status, experiences, and perceptions of learners and teachers. 

7. The concept of literacy is based on the teachers' and learners 1 values 
and beliefs. 

These seven statements reflect the staff members’ underlying 
assumptions about literacy. From this initial framework, several 
implications can be drawn. First, literacy should be developed in ways 
that are meaningful to learners. Second, the literacy curriculum needs 
to address the ways in which learners want to use literacy. Third, 
learners should be deeply involved in the development of the curricu- 
lum content. Fourth, literacy instruction is a collaborative effort 
between the learner and the teacher. 

In an actual project, the literacy framework would be refined as 
the project is implemented and evaluated. Initial aspects of the frame- 
work might be quite general; however, as the staff and learners con- 
tinue to work together, they will improve their understanding of the 
nature of literacy and how it can be fostered in the project. 

To support a literacy framework such as the one described above, 
assessment and evaluation procedures should do the following: 

1. Focus on the learners — that is, accurately identify learners’ 
strengths and needs and assess learners’ progress in the project; 

2. Document how improvement in learners’ literacy development 
affects their children, work responsibilities, and community 
activities; 

3. Rely on a variety of instruments and strategies for collecting data; 

4. Reflect changes in learners’ perceptions, attitudes, and behavior 
inside and outside the classroom; 

5. Collect and analyze data on a continual basis throughout the project. 

Process and Product Evaluation 

Alternative assessment ar.d evaluation procedures raise a number 
of questions related to process and product evaluation. A process 
evaluation focuses on how a project is implemented. A question that 
might be addressed in a process evaluation would be the following: 
How and to what extent are learners involved in identifying their 
literacy goals and objectives, designing the learning activities of the 
project, and assessing their progress in attaining their goals? 

A product evaluation is concerned with the effect of the project 
on the adult learners. A product evaluation addresses such questions as 
the following: To what extent have adult learners increased their 
language proficiency? To what degree have they attained their per- 
sonal literacy goals? To what extent does increased literacy develop- 
ment affect how learners (1) perceive themselves, (2) relate to their 
spouses or other family members, (3) respond to friends and co- 
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By using a variety of 
alternative approaches, staff 
and learners can develop a 
comprehensi 9 picture of what 
is happening In the project and 
satisfy the needs of both 
process and product 
evaluation* 



Developing a quality 
instructional program and 
evaluation design requires a 
thorough planning process 
involving the collaboration of 
staff members and adult 
learners. 



Effective planning is not 
a one-time effort, but rather a 
series of activities that occur 
throughout the life of the 
project. 



workers, and/or (4) use their literacy skills in new settings or for 
different purposes? 

To answer these and other questions, learners need to be assessed 
in different ways. The alternative approaches presented in Chapters III, 
IV, and V include sur\*eys, interviews, obser\>aiion measures, and 
performance samples . Surveys and interviews of family members, 
friends, co-workers, employers, and classmates can provide valuable 
information regarding learners’ needs, goals, and preferences for 
curriculum content. Observation measures can be used by staff to 
assess the learners’ progress in acquiring language and literacy. 
Performance samples of learners’ work, such as writing samples, tape 
recordings of oral readings, and journals, are useful for documenting 
learners' language development. By using a variety of alternative 
approaches, staff and learners can develop a comprehensive picture of 
what is happening in the project and satisfy the needs of both process 
and product evaluation. 

A Planning Process Mode! 

Developing a quality instructional program and evaluation design 
requires a thorough planning process involving the collaboration of 
staff members and adult learners. Figure I illustrates each of the steps 
of the process. The model is heuristic; that is, it is shown as an ideal- 
ized guide for planning. In actual practice, the tasks are not always 
implemented in sequence. Sometimes steps are skipped and then gone 
back to. At other times, steps are addressed simultaneously. Most 
important, the steps are interdependent; that is, what happens in 
implementing one step influences activities in other steps. 

The planning process is cyclical; the knowledge gained by 
implementing one step is used to inform decisions related to other 
steps. The implementation of the steps should be a collaborative effort 
between staff and adult learners. Figure 1 illustrates that effective 
planning is not a one-time effort, but rather a series of activities that 
occur throughout the life of the project. By continuously examining the 
needs of the learners, staff, and the community in general, staff mem- 
bers can create instructional activities, assessment strategies, and 
evaluation procedures that are highly responsive to the learners and the 
conditions under which the project is being implemented. 

Staff and learners need time to organize and implement the steps 
in the model. Although the entire process cannot be implemented fully 
in any one year, staff and learners should work through as many steps 
as possible, implementing fully when feasible and laying the ground- 
work for other steps that might be addressed later. 

The process is designed to give staff and learners a blueprint for 
continuously collaborating to design and refine their project. Visualiz- 
ing the entire process enables staff to schedule time and resources for 
dealing with each step. For example, teachers may devote class time to 
activities in which learners would discuss the relevance of instructional 
content to their personal goals and the extent to which they believe 
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they are making progress. Ideally, staff members should be able to 
meet weekly or biweekly to assess and clarify goals, objectives, 
instructional activities, and other issues. 

Staff members will benefit from planning activities that result in 
( t ) clarifying adult learners' needs, (2) defining the dimensions of 
literacy addressed by the project, (3) refining curriculum content, and 
(4) selecting appropriate instructional methods. By establishing a 
consensus about the important aspects of the project, staff members 
can coordinate their responsibilities more successfully. They also can 
provide a clear orientation to new staff members and adult learners 
who join the project in midstream. A literacy framework becomes a 
useful reference for communicating the program's philosophy, goals, 
and procedures to others. Specifying clear goals and objectives helps 
participants collaborate to create a consistent, coherent instructional 
program that responds to learners' needs and supports their continued 
growth. A collaborative planning process enables participants to 
identify changes that will improve the project based on the collective 
resources of staff and learners. 



Positive interpersonal 
relations can be enhanced by 
creating activities designed to 
build a cooperative spirit 
among staff and adult 
learners. 



Applying the Planning Process Model 

Collaboration is facilitated when individuals establish a trusting, 
supportive relationship with each other. Staff members need to model 
the cross-cultural sensitivity that is essential for developing positive 
relationships. Positive interpersonal relations can be enhanced by 
creating activities designed to build a cooperative spirit among staff 
and adult learners. Team-building exercises are useful for developing 
participants' ability to recognize team. members' contributions and 
realize the value of helping and receiving help from one another. 
Developing cooperative skills will improve the participants' ability to 
implement the planning process model. (For more on improving 
human relations in the work piace, see Johnson, 1987.) Chapter IV 
contains procedures for assessing adult learners' group participation 
skills. 

The following are descriptions of each step of the planning 
process model and examples of how the step has been addressed by 
FELP projects in California. These, descriptions represent only some of 
(he many approaches that are possible for designing, implementing, 
and evaluating family literacy projects. 

1. Develop the goals of the project. 

Goals are broad, general statements of the intent, purpose, or 
expected outcome of a project. Usually the goals of a literacy project 
are derived in part from the purposes of the program as delineated by 
the ESEA Title VII statute for FELP. 

Goal statements help staff members design the needs assessment 
activities that take place in Step 2 of the planning process. In the initial 
stages, goals are tentative, because they may need to be adjusted based 
on what the staff learn horn assessing the needs of the learners. As the 
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The goals represent the 
focus of the project and reflect 
the choices that the project 
staff have made within the 
flexibility of the funding 
source and the needs of the 
participants as anticipated by 
the staff. 



The first phase of the 
needs assessment usually is 
conducted as part of the 
proposal for funding. 



project is implemented (Steps 4-12), the staff may need to adjust further 
the goals of the project. 

The goals developed by the FELP project at the Santa Clara 
County Office of Education (1989-92) address the major purposes of 
FELP; that is, developing the participants’ English language skills and 
the parents’ ability to help their children succeed in school. The follow- 
ing goals were designed to reflect the unique interests of the learners 
and staff in the Santa Clara area {Proceedings from the Family English 
Literacy Seminar, 199 1 , p. 62). 

1. Assist LEP Hispanic and Southeast Asian parents and family members to 
achieve literacy and competence in the English language; 

2. Provide parents and family members with knowledge of techniques 
designed to assist their children in developing literacy skills in English 
and appreciation of reading; 

3. Involve parents and family members in direct instructional and support 
roles related to their children’s skill development and participation in 
English literacy; and 

4. Support parent/community/school communication and the development 
of parents' leadership, advisory, and advocacy roles. 

The goals represent the focus of the project and reflect the choices 
that the project staff have made within the flexibility of the funding 
source and the needs of the participants as anticipated by the staff. 

2. Conduct the needs assessment. 

Assessing the needs of prospective learners is a vital step in the 
planning process. During this step, staff obtain valuable information 
about the learners, their families, and their communities. The first phase 
of the needs assessment usually is conducted as part of the proposal for 
funding. The questions asked of learners are based on the goals that 
staff intend to pursue in response to the funding guidelines. After 
receiving the grant, staff members need to conduct more in-depth needs 
assessments, the results of which may cause staff to clarify or modify 
the goals. 

Before the FELP project in the Lincoln Unified School District 
(1990-93) designed its objectives and instructional activities, staff 
administered a survey (alternative assessment instrument) to a repre- 
sentative sample of the Southeast Asian families that the district 
intended to serve. (Chapter III contains a detailed explanation of the 
needs assessments that were conducted by staff at Lincoln.) The results 
of the survey indicated that the parents wanted services to be available 
near their apartment complex. Therefore, the staff established a family- 
education center in a vacant apartment unit located in the middle of a 
two-block area of five apartment complexes occupied by Cambodian 
and other Southeast Asian families. 

The staff of the FELP project at the National Council of La Raza 
(1988-91) used writing samples produced by project participants as part 
of its needs assessment. The adult learners wrote the sample in Spanish 
or English, depending on their language proficiency. The staff used a 
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information gathered 
through needs assessment 
activities can help staff to de- 
sign project objectives (Step 3} 
and develop curriculum and 
instructional activities (Step 4). 



Each objective should 
include procedures and 
instruments that can be used 
to measure the extent to which 
the objectives are being 
achieved. 



Objectives are 
determined by what the 
learners need, what the staff 
are prepared to teach, and 
what is possible to achieve 
within the time and budget 
constraints of the project. 



holistic scoring technique to score the learners’ writing. The samples 
provided important information for refining objectives, developing 
instructional activities, and generating baseline data for comparing and 
evaluating changes in the learners’ writing ability at a later time in the 
program. Chapter IV contains more information about using writing 
samples for assessment. 

The FELP project in the Whittier Union High School District 
(1990-93) used data obtained from the academic records of the adult 
learners’ children to design objectives and instructional activities. 

These data enabled staff to identify high school students who were 
experiencing academic problems and to design project activities that 
would help parents improve their children’s academic performance. 
Information gathered through needs assessment activities can help staff 
design project objectives (Step 3) and develop curriculum and instruc- 
tional activities (Step 4). 

3. Design instructional objectives. 

Objectives can help staff specify changes or outcomes that 
learners may experience in their language abilities, parenting strate- 
gies, cultural adaptation, or other important areas as a result of the 
instructional program. Some objectives may deal with clearly observ- 
able outcomes such as speaking and writing; others, however, may 
specify changes that are more difficult to see such as self-confidence, 
group participation, or literacy awareness. Each objective should 
include procedures and instruments that can be used to measure the 
extent to which the objectives are being achieved. (Chapter I contains 
additional information about objectives.) 

Initially, objectives are based oh information collected during the 
needs assessment. As the planning process unfolds and more informa- 
tion is collected from the learners, objectives and related assessment 
instruments may need to be refined. Ultimately, objectives are deter- 
mined by what the learners need, what the staff are prepared to teach, 
and what is possible to achieve within the time and budget constraints 
of the project. 

The following are examples of objectives that use alternative, 
nonstandardized assessment measures ( Family English Literacy 
Program: ESEA Title VII Application , 1991, pp. 26-29). 

1. By the end of each project year, participants will improve their recep- 
tive (listening and reading) and productive (speaking and writing) skills 
at least one level as assessed by an observation measure and writing 
sample. (This objective contains four subobjectives that specify 
outcomes and measurements for listening, reading, speaking, and 
writing.) 

2. By the end of each project year, participants will increase by at least 
four the number of literacy activities they do with their children as 
assessed by individual interviews. 
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4. Develop the curriculum and instructional activities. 

Instructional activities arc based on the learners’ needs and the 
project’s goals and objectives. For example, the staff of the FELP 
project in the Parlier Unified School District (1989-92) responded to 
the large numbers of Spanish-speaking families in the district by 
designing activities in which parents use original stories from Latin 
American countries for storytelling activities with their children. 

The FELP staff of the Fremont Unified School District ( 1988-91) 
organized instructional activities for parents that used children’s 
stories from the district's adopted list of core literature. By becoming 
aware of the literature that their children studied in school, the parents 
gained confidence in talking with their children about school activities. 
Chapter IV contains more suggestions for developing instructional 
activities and designing ongoing assessment instruments and proce- 
dures. 

5. Identify obstacles. 

’‘Forewarned is forearmed" accurately describes the value of 
anticipating potential obstacles to planned activities. Knowing about 
possible impediments helps staff and learners anticipate and refine 
future activities by modifying plans and elaborating strategies to 
overcome hurdles. At a coordination meeting of 14 California FELP 
projects in December 1990, participants identified obstacles they faced 
with regard to 1) recruitment of participants, 2) maintaining enroll- 
ment, and 3) staff recruitment and training. Next, they shared strategies 
they had found successful in addressing these problems. The following 
are results of their discussions [Proceedings from the Family English 
Literacy Seminar, 1991, pp. 15-20). 

1. Obstacles to recruiting participants: 

a. Schools have incomplete or inaccurate lists of limited-Knglish- 
profieient students. 

b. Printed flyers advertising classes arc ineffective because parents may 
not be literate in English or their native language. 

e. Attending school may be a new experience for parents who have little 
or no education in their native countries. 

d. Parents are reluctant to visit the school because they arc not knowl- 
edgeable about the educational system. 

e. Students who qualify for the Greater Avenues for Independence 
(GAIN) program may not participate in the project if classes do not 
meet GAIN requirements. 

f. Some parents face cultural barriers that restrict their participation, for 
example, in some cultures, wives cannot attend class without their 
husband's permission. 

g. Child care is not available. 

2. Obstacles to maintaining enrollment: 

a. Participants must move frequently because of seasonal jobs. 

b. Participants’ mertime work presents them from attending classes. 



c. Responsibilities at home prevent regular attendance — lor example, 
child care, family problems, and transportation. 

d. Participants may become discouraged if the project activities do not 
meet their needs. 

3. Obstacles to staff recruitment and training: 

a. Staff who are knowledgeable about adult ESL, parenting, and lan- 
guage-minority issues are in short supply. 

b. Qualified bilingual teachers are in limited supply. 

e. Some staff are reluctant to work in the evening, 
d. Budgets are insufficient to support staff salaries. 

By identifying obstacles and discussing possible solutions, 
participants can focus on alternatives to consider for overcoming 
significant barriers to improving their projects. 

6. Identify resources needed to address each objective and 
obstacle. 

Staff need to identify the resources necessary to address objec- 
tives and obstacles. Resources include personnel, time, materials, ar.d 
funding. This step helps staff analyze the scope of the project and 
establish realistic expectations for what can be accomplished in the 
project. 

7. Assign staff members responsibility for designing and 
implementing each task or activity. 

When staff members are responsible for particular tasks, they 
become stakeholders who regard their contributions as important 
factors in the success of the project. Accountability is especially 
important in collaborative efforts to ensure that each team member 
contributes. 

8. Establish a time line for each task or activity. 

Because family literacy projects offer a broad array of services 
and activities over an extended period of lime, staff members need to 
specify a time line to ensure that all steps of the planning process are 
implemented. Steps 6-8 are essential because they provide the param- 
eters for developing each activity. 

Examples of Steps 6-8, 

When staff members of the FELP project located at California 
Slate University, Sacramento ( 1986-89) identified resources needed to 
improve the project (Step 6), they determined that having their own 
curriculum would significantly facilitate the learners' achievement of 
the project’s objectives. During the first year of project implementation, 
the staff compiled curriculum content and activities that the learners 
especially liked. Each staff member assumed a specific role for design- 
ing. compiling, and evaluating the curriculum materials (Step 7). The 
adult learners also had responsibilities such as identifying content they 
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The adult learners also 
had responsibilities such as 
identifying content they 
believed was important to 
learn and giving feedback 
regarding the effectiveness of 
learning activities. 




Staff members and 
learners may want to develop 
outcomes that are short term 
(the ability to explain one's 
needs to an apartment 
manager) and long term 
(obtaining a teaching job). 



believed was important to learn and giving feedback regarding the 
effectiveness of learning activities. After deciding on the scope of the 
curriculum, the staff created a time line for developing the curriculum 
for the project (Step 8). 

During the second year, the staff field tested and refined the 
curriculum materials. By the end of the third year, the curriculum was 
printed and used at all sites in the project. Titled Parenting Curriculum 
for Language Minority Parents (Holt, 1988), the text was developed in 
English, Chinese, Hmong, Khmer, Korean, Lao, Spanish, and Viet- 
namese. A teacher’s activity guide also was developed. The curriculum 
is an example of how staff can collaborate with learners to identify and 
develop resources that enhance the implementation of the project. The 
success of the effort was dependent on ( 1 ) staff members’ having an 
intimate knowledge of the adult learners’ needs, (2) staff and adult 
learners’ sharing responsibilities for tasks, and (3) staff members’ 
adhering to a time line for completing the tasks. 

Parenting Curriculum for Language Minority Parents has also 
been a valuable resource for FELP project staff of the Stockton Unified 
School District (1988-91). Staff used the curriculum to improve 
parents’ knowledge of such topics as parent-child communication, 
child development, discipline techniques, and the U.S. educational 
system. Each topic, however, was addressed only after the staff and 
learners agreed that it was important. 

Staff of the FELP project in the Ravens wood City Elementary 
School District (1989-92) found that computers were an important 
resource for enhancing the implementation of their project (Step 6). 
The computers were used to supplement the literacy instruction the 
parents received in Spanish and English. For example, beginning-level 
learners composed stories on the computer, and advanced learners 
created their own resumes and developed a project newsletter. The 
computers became a resource for motivating parents to participate in 
project activities and for strengthening the relationship between par- 
ents and the » children. 

9. Specify expected outcomes. 

During this step of the planning process, staff and learners iden- 
tify the type and amount of change or growth they expect at the end of 
a given time period. This step represents what the project participants 
regard as success in the project. Growth can be expressed in terms of 
academic growth (for example, a specified increase in positive ratings 
on a project-developed reading test); social, cognitive, or affective 
development; organizational skills; or parenting skills. The outcome 
statements are more detailed than the objectives that were developed 
earlier in the process (Step 3). They should specify the assessment 
measures to be used and the amount of growth that seems realistic. 
Staff members and learners may want to develop outcomes that are 
short term (e.g., the ability to explain one’s needs to an apartment man- 
ager) and long term (e.g., obtaining a teaching job). 
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10. Design assessment instruments and procedures to 
collect data. 

Staff and adult learners need to identify the data that will docu- 
ment progress toward attaining (he outcomes specified in Step 9. For 
example, taking a frequency count of the number of times children read 
to parents per month might be an indicator of parents’ encouragement 
of children’s literacy development. 

Outcomes may be assessed with various types of assessment strat- 
egies. For instance, an outcome such as growth in listening comprehen- 
sion could be assessed through a single measure such as a standardized 
test, an interview, an observation measure, or a performance sample. 
However, an outcome that deals with reading strategies ilia' an adult 
learner uses to obtain information might be assessed through a combi- 
nation of an interview, an observation measure, and a performance 
sample to fully capture all the strategies that the reader might be u; ing. 
To gather data accurately, it is important that a match be established 
between the lype(s) of assessment and the nature of the outcome. A 
variety of alternative procedures for assessing specific outcomes are 
discussed in Chapters 111, IV, and V, 

11. Analyze data. 

Data become meaningful when they are analyzed. Various ap- 
proaches can he used for data analysis. Staff may want to represent 
assessment data in numerical scores to generate percentages, means, 
and standard deviations. On the other hand, staff may wish to describe 
learner outcomes by summarizing assessment results in narrative form. 
To interpret and analyze data effectively, staff members need lo under- 
stand thoroughly the outcomes that were assessed. Chapter V contains 
detailed suggestions for analyzing data. 

12. Formulate conclusions and implications. 

The results of the data analyses should be reviewed by staff and 
learners to determine the extent lo which they accomplished what they 
set out to do. That is, were the objectives of the program attained? Were 
the outcomes specified in the objectives realistic? Were they matched to 
the learners' needs? Should instructional activities be changed? Should 
objectives be modified? This step provides staff and learners with the 
opportunity to make what are often hard choices about the project. 

What has been successful? What needs to be strengthened? What ought 
to be eliminated? What need'- lo be changed? 

As shown in Figure 1, the decisions made in Step 12 should be 
used lo refine the activities associated with Steps 1, 2, 3, and others. 
Conclusions about the outcomes of project activities should influence 
the development of objectives that will be pursued in the future. That is. 
the planning process model represents a cycle of interdependent steps. 
When any aspect of the program is changed, all of the related aspects 
should be reviewed to determine if they also need to be modified. Chap- 
ter V provides a variety of strategies for using and reporting data. 
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A planning team made 
up of staff members and 
learners will help ensure that 
decisions are based on a 
thorough understanding of the 
needs of all participants. 



Because of the diversity 
of individuals and the range of 
perspectives that need to be 
blended into a consensus, it is 
vital that the facilitator of the 
planning team be both willing 
and able to promote a 
cooperative spirit among team 
members. 




Designing Alternative Approaches to Assessment 

and Evaluation: Where Do You Begin? 

The preceding section presented a process for planning, imple- 
menting, and evaluating a family literacy project. The model empha- 
sizes the importance of collaboration among the staff members and 
learners for creating a successful project. The planning process also 
underscored the value of alternative assessment procedures for helping 
staff ascertain the needs of adult learners and the progress they arc mak- 
ing in the project. 

The final section of this chapter presents suggestions for imple- 
menting the planning process model and related alternative approaches 
to assessment and evaluation. Implementing the model involves decid- 
ing ( 1 ) who will be involved in the planning process, (2) how the plan- 
ners will work together, and (3) what issues the planners v ill address in 
assessment and evaluation. 

Members of the Planning Team 

The first and perhaps most important step in initiating a compre- 
hensive planning process is to decide who will be involved in the pro- 
cess. A planning team made up of staff members and learners will help 
ensure that decisions are based on a thorough understanding of the needs 
of all participants. The project evaluator might also be included in the 
planning team to assist the staff and learners with designing the evalua- 
tion. 

To the extent possible, all instructional staff should have the op- 
portunity to participate in the planning process. Community liaisons also 
should participate because of their knowledge of the learners’ language 
and cultural backgrounds. When deciding on the content of the curricu- 
lum and specifying learner outcomes, it may be helpful to include sug- 
gestions from prospective employers, teachers of the participants* chil- 
dren, representatives from local government agencies (for example, 
employment and social service offices), and staff from community col- 
leges and universities. 

Approaches to Planning 

After the planning team has been formed, team members need to 
consider how they can work together effectively. Because of the diver- 
sity of individuals and the range of perspectives that need to be blended 
into a consensus, it is vital that the facilitator of the planning team be 
both willing and able to promote a cooperative spirit among team mem- 
bers. The facilitator needs to ensure that each team member's contribu- 
tions are invited, expressed, valued, and integrated into the decisions 
that arc made in each step of the planning process. 

The facilitator has the responsibility of ensuring that meetings and 
tasks are structured so that the project benefits from the experiences of 
all team members. A collaborative approach to planning enables staff 
and learners to become more knowledgeable about each other's needs 
and the value of cooperation itself. According toFreire (1973). “acquir- 
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ing literacy does not involve memorizing sentences, words, or 
syllables — lifeless objects unconnected to an existential uni- 
verse — but rather an attitude of creation and re-creation, a self- 
transforrnation producing a stance of intervention in one’s con- 
text” (p. 48). Collaboration helps team members develop a sense 
of responsibility for helping others and the feeling of satisfaction 
that comes from a successful team effort. The act of planning and 
implementing together can provide significant opportunities for 
learning how to assess what one needs and how to take steps to 
improve one’s context, whether it be at home, at school, or at 
work. 



The planning team 
needs to decide what data 
should be collected, analyzed, 
used, and reported to provide a 
comprehensive picture of the 
implementation of the project. 



To design the product 
evaluation, the most important 
point for the planning team to 
address is the specification of 
what constitutes success in 
the project. 



Issues in Assessment and Evaluation 

The primary purpose of the planning team is to take the 
necessary steps for designing, implementing, and evaluating the 
project. This section presents questions that the planning team 
needs to answer related to designing the assessment and evalua- 
tion of the project 

1. What should be evaluated? 

Staff members need to consider how they will collect data 
for assessing the implementation of the project (process evalua- 
tion) and the effects of the project on the learners’ lives (product 
evaluation). To design the process evaluation, the planning team 
should deal with such issues as the role of the staff members and 
learners in planning, implementing, and evaluating the project. 
Other points to examine might be the extent to which the goals of 
Ihe project are consistent with the goals of the learners. The plan- 
ning team needs to decide what data should be collected, ana- 
lyzed, used, and reported to provide a comprehensive picture of 
the implementation of the project. 

Product evaluation, on the other hand, deals with issues 
related to the effects of the project on adult learners (and their 
children, if the objectives cf the project include children). To 
design the product evaluation, the most important point for the 
planning team to address is the specification of what constitutes 
success in the project. In other words, how will the staff and 
learners know the extent to which the goals and objectives of the 
project are being met? How will participants determine that 
learners have increased their ability to use language for various 
purposes or their self-confidence in working with others? What 
data will be used to make these determinations? How will the 
data be collected? 

To determine the value of collecting any piece of data, the 
planning team needs to clarify the purpose of the data and ana- 
lyze whether the cost of collection is worth the required lime and 
resources. For example, will the data be used to improve initial 
assessment, to enhance the curriculum, to show learners the 
progress they are making, to develop staff members’ teaching 
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ability, to document progress for outside agencies, or to improve the 
administration of the project? 

2. When should data collection take place? 

After determining what data will be collected, the planning team 
needs to decide when the data will be gathered. Questions to address 
include the following: Are the data needed before the learners enroll in 
the project? How many times should data be collected to show 
progress? Chapters III and IV contain information about the timing of 
initial and ongoing assessment procedures. 

3. How will adult learners, staff members, the project evaluator, 
members of the community, and others be involved in conducting 
the evaluation? 

The planning process model emphasizes the importance of col- 
laboration in helping to ensure an evaluation design that is comprehen- 
sive and meaningful. The planning team needs to consider who should 
be involved in the evaluation, the role of each individual, and strategies 
for facilitating a cooperative effort. 

4. What data collection procedures will be used? 

The planning team should decide what procedures will be used to 
collect the data. For example, what data will be gathered via alternative 
instruments such as surveys, interviews, observation measures, and 
performance samples? How will standardized measures be used? Fac- 
tors to consider in these decisions are the comparative effectiveness of 
each procedure in providing accurate and reliable data, resources 
needed for developing the procedures, staff members’ knowledge or 
prior experience in using the procedures, and time and resources neces- 
sary for collecting and analyzing the data. 

5. How will staff members know that progress is being made? 

Objectives and expected outcomes of the project represent the 
staff members’ criteria for project success. For most family literacy 
projects, criteria include a range of outcomes such as gains in listening, 
speaking, reading, and writing abilities; improved self-confidence; en- 
hanced use of literacy in different contexts; increased abilities of par- 
ents to help their children in school; improved involvement of parents in 
their children’s school activities; and greater use of community re- 
sources. Chapter IV provides examples of other learner outcomes and 
related alternative assessment procedures. 

6. How will stakeholders and others become knowledgeable about 
the results of the evr Kiation? 

One of the uses of the evaluation results is to inform stakeholders 
of the nature of the literacy project and the progress th t the learners are 
making. Stakeholders may include the learners in the project, other staff 
members, administrators, family members, and representatives of the 
funding agency. The planning team should determine who needs to re- 
ceive the evaluation results and how the results will be disseminated. 
Chapter V provides examples of strategies for reporting the results of 
the evaluation. 



Assessing Success 



42 




When assessment and 
evaluation are integrated with 
instruction, they produce 
results that can be used to 
document success and plan 
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Summary 

Traditional standardized assessment and evaluation procedures 
provide only some of the answers to the questions that staff and others 
may have about family literacy projects. The standards on which tradi- 
tional procedures are based are not always tailored to the needs, goals, 
and characteristics of the diverse adult learners who participate in lit- 
eracy projects. In contrast, alternative approaches to assessment and 
evaluation enable project staff to create instruments and procedures that 
provide highly useful and meaningful information for improving their 
project. When standardized and alternative assessment procedures are 
integrated, they can provide a more complete understanding of the lit- 
eracy project — its objectives, activities, and results — and help meet the 
information needs of the learners, program staff, and funding agency. 

To be effective, alternative approaches should be linked carefully 
to the needs of the adult learners, literacy framework, objectives, and 
curriculum content of the project. Such linkage can be achieved by using 
a planning model that is designed and implemented through the collabo- 
rative efforts of staff and learners. The collective commitment of all 
project participants to a comprehensive planning process will help 
ensure that assessment and evaluation activities are implemented as an 
integral part of planning and implementing the project. When assess- 
ment and evaluation are integrated with instruction, they produce results 
that can be used to document success and plan improvements in the 
project. 
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Initial assessment is 
important in the development 
of a successful literacy project 
because it provides program 
planners with the opportunity 
to examine the needs, desires, 
and goals of the potential 
participants. 



Objectives are derived 
from the identified needs; 
activities are designed to 
achieve the objectives; and 
assessment procedures are 
created to evaluate the 
attainment of the objectives. 



initial Assessment in 
Planning a Successful Project 

This section focuses on initial assessment as an integral part of 
the planning of a family literacy project — Step 2 in the Planning 
Process Model introduced in Chapter II. The following issues will be 
addressed: ( 1 ) the importance and nature of initial assessment in the 
planning of a family literacy pro ject and (2) approaches that can be 
used for initial assessment. 

Importance and Nature of Initial Assessment 

Initial assessment is important in the development of a successful 
literacy project because it provides program planners with the opportu- 
nity to examine the needs, desires, and goals of the potential partici- 
pants. This information can be used to generate the instructional 
objectives and other aspects of the program design. Figure 1 illustrates 
how the results of a needs assessment influence the key aspects of the 
project. 

An effective needs assessment can help staff understand the 
needs and goals of the participants and establish the foundation for the 
entire project. Objectives are derived from the identified needs; 
activities are designed to achieve the objectives; and assessment 
procedures are created to evaluate attainment of the objectives. (The 
effects of initial assessment are illustrated in more detail in “Planning 
Process Model,” Figure I in Chapter II of this volume.) 
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A family literacy project should be planned on the basis of 
extensive information obtained about the learners. According to 
Weinstein-Shr (1993), project staff may wish to learn about ( 1 ) the 
structure and characteristics of the community in which the {earners 
live; (2) the language, literacy, and educational background of the 
learners; and (3) the concerns of adults in their role as parents. The 
following list expands these issues and presents a wide range of 
information for staff to consider in designing a needs assessment: 

1 . Demographic information about the learners’ community (for example, 
number of language groups, length of residency, and degree of transiency). 

2. Historical and cultural backgrounds of the learners (for example, country 
of origin, obstacles faced in co' ng to the United States, family structure, 
and religious affiliation). 

3. Information about family members (for example, average number of 
family members, socioeconomic level, and literacy levels in English and 
their native language). 

4. Educational, parenting, career, psychosocial, and health needs of the 
parents (for example, educational background, reasons for learning 
English, educational expectations for children, vocational goals, and 
support network). 

5. Educational, psychosocial, and health needs of the children (for example, 
educational background, language proficiency, and health history). 

6. Determination of which needs are short t c t m and which are long term (tor 
example, the learners’ psychological nec-.^ that require immediate 
attention versus their interest in attending college). 

Other needs may be identified by consulting with teachers, administra- 
tors, service providers, and others in the community. 

Approaches for Initial Assessment 

Initial assessment to determine the needs of potential project 
participants may involve the use of surveys , interviews, and focus 
group discussions with community members as well as analysis of 
documents and literature regarding family literacy. Each of these 
approaches is described below, with examples of how they were 
carried out for a needs assessment in the Lincoln Unified School 
District in Stockton, California. An in-depth explanation of the process 
of analyzing and reporting the results of these approaches is given in 
Chapter V. 

Surveys. Surveys are effective for assessing the needs of potential 
project participants. The survey may consist of a written questionnaire 
or check list that is completed by prospective participants or adminis- 
tered orally by bilingual staff members to individuals in English or 
their native language. Responses to closed-ended questions may be 
easier to tally, but open-ended questions can give staff a more com- 
plete understanding of the learners 1 needs. At the beginning of the 
planning process, a survey can be administered to a large group of 
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individuals to create a general picture of the needs of adults and other 
family members who may be interested in project services. 

For example, in 1990 me staff of the Lincoln Unified School 
District developed a survey shown in Example 1 (see page 50) to 
collect basic information from prospective participants regarding their 
need for a family literacy project. The survey addresses such issues as 
number of adults and children residing in each housing unit; their 
native language; number of adults who would like additional English 
instruction; preferred location and time of literacy classes; parents' 
need for child care and transportation; desired curriculum topics; self- 
assessment of English language proficiency; number of years' resi- 
dence in the United States; and number of children who are enrolled in 
ESEA Title VII projects. To obtain more detailed information about 
what the participants would like to learn in the project, staff could use 
a survey such as the one shown in Example 2 (see page 51). 

To get the most comprehensive and valid results, four bilingual 
staff members of the district visited 30() apartment units during a two- 
week period to administer the survey orally to individual residents. 

This instrument, and others shown in the chapter, can be administered 
in English or the participants’ native *anguage. The survey results 
indicated that the Cambodian adults living in the targeted area were 
especially interested in services that a family literacy project could 
provide. For eample, parents wanted to know how they could help 
their children learn Khmer and how they could volunteer in the 
schools. Staff also realized that the parents' limited access to transpor- 
tation made it necessary to have literacy classes located near their 
apartments. On the basis of those findings, the staff decided to estab- 
lish a family education center in a vacant apartment located in the 
middle of a two-block area of five apartment complexes occupied by 
Cambodian families. The survey also helped staff determine the most 
convenient times for literacy classes, the literacy needs of the prospec- 
tive participants, and their related needs, such as counseling and 
cultural adaptation. 

A door-to-door survey such as this one is practical when the 
prospective participants live in a concentrated area. If families are 
dispersed, bilingual staff can administer the survey via telephone. 
Because the Vietnamese families live throughout the Lincoln district, 
bilingual staff used the telephone to administer an adapted version of 
the survey shown in Example 1. The needs of the Vietnamese families 
differed significantly from those of the Cambodians. Rather than 
attending literacy classes in their neighborhood, the Vietnamese adults 
preferred to go to one of the district’s school sites for classes. They 
also indicated their desire for citizenship classes, whereas the Cambo- 
dian adults did not express this interest, a difference that became 
understandable in light of other kinds of information gathered during 
the needs assessment process. For example, almost all the Cambodians 
surveved were forced to leave their country by the Communist Khmer 
Roug uid looked forward to returning in the future, whereas most of 
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help staff members gather 
information from a large 
number of people in a short 
amount of time. 



interviews are effective 
for obtaining detailed 
information from individuals. 



the Vietnamese adults chose to come to the United States for 
political freedom and educational opportunity. These major 
differences between the two community groups highlight the 
crucial importance of conducting a needs assessment when 
planning a project. 

Focus groups. Focus group discussions help staff mem- 
bers gather information from a large number of people in a short 
amount of time. A focus group discussion is conducted with 
small groups according to a protocol that focuses on specific 
topics, procedures, and questions. The goal of analyzing focus 
group data is to examine systematically the participants’ com- 
ments to identify trends or patterns that recur among different 
individuals or groups. As part of the initial nt Is assessment for 
its family literacy project, staff of the Lincoln Unified School 
District conducted focus group discussions with members of the 
district’s bilingual advisory council; potential project partici- 
pants; the participants’ children; and representatives of the 
school district, including teachers and administrators. 

Interviews. Interviews are effective for obtaining detailed 
information from individuals. Lincoln’s staff interviewed 
service providers in the community, such as librarians, and 
visited family literacy projects located in northern California to 
determine how the literacy and social needs of Southeast Asian 
families were being addressed by various service providers and 
the relative effectiveness of those services. The interviews 
provided yet another view' of the learners* needs and an under- 
standing of the level of assistance available to meet the needs. 
The information also helped identify content and methodology 
for literacy classes and strategies for coordinating project 
services with other community agencies. 

Document and literature review. Learners' needs also can 
be assessed by analyzing documents and literature related to 
family literacy. Data can be gathered from school district 
records, including test scores, grades, and attendance records of 
participants’ children; welfare department statistics such as the 
socioeconomic status of participants; and journals, newsletters, 
books, and state and national reports that include information 
about the literacy-related needs of participants. 

Staff in the Lincoln district analyzed demographic data and 
statistical reports regarding Southeast Asian families. For 
example, staff identified the number of Cambodian and other 
Southeast Asian families living in San Joaquin County and then 
projected the number that would be residing in the district in the 
future. In addition, they consulted the Report to the Congress: 
Refugee Resettlement Program (IJ.S. Department of Health and 
Human Services, 1989), which indicated that the reading and 
employment levels of Cambodian adults in the district were 
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significantly below the national averages for other Southeast Asians. 

The combined findings from .surveys, focus groups, interviews, 
and research reports constitute a thorough assessment of the need fora 
family literacy project. After gathering the information, project staff 
should analyze results, identify commonalities, and develop a compre- 
hensive summary that reflects all the needs of the targeted participants. 
The next step is for stuff to select those needs that will be the focus of 
the project and that can realistically be met with available resources. 
Funding agencies recognize that not all needs can be met by any one 
project and that the scope of a grant proposal should be limited and 
clear. 

A thorough needs assessment enables staff to address the next 
steps of the planning process: that ( I ) developing detailed program 
objectives that respond to the needs, (2; identifying the curriculum and 
instructional activities that wall lead to the achievement of each 
objective, and (3) developing assessment procedures to measure the 
achievement of each objective and generate data for a final evaluation 
report. (See Chapters 1 and II for more information on developing 
project objectives.) Before moving on to these steps, however, staff 
and prospective learners should develop a consensus regarding the 
philosophical basis of instructional methods, curriculum content, and 
other aspects of the project. Time should he devoted to the develop- 
ment of the project's literacy framework. 

Developing a Literacy Framework 

Staff members in many family literacy projects have found it 
helpful to develop a shared understanding among staff and learners 
regarding key aspects of the project. This shared understanding is 
referred to in Chapter II as the project's “literacy framework. “ Such a 
framework will facilitate discussions and decisions about the needs 
assessment, program design, and evaluation. Issues to discuss include 
( 1 ) learners' literacy needs. (2) goals and objectives of the project, (3) 
curriculum content and instructional methodology, and (4) approaches 
for assessing and evaluating the learners' literacy development. 
Developing consensus on such complex issues is a challenging task, 
and staff and learners may vary in the extent to which they are able or 
willing locollaborate to reach, common ground. It is important, how- 
ever, for all participants to make an ongoing commitment to work 
together and to draw on each other's expertise. 

As general issues become resolved, staff and learners can pursue 
more detailed aspects of the project, such as the following: 
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1 . Role of literacy in the learners' daily lives. 

2. Relative importance of the goals and objectives of the project. 

3. Curriculum content and instructional methodologies, including the use of 
the learners’ native language and English in the project. 

4. General design of the assessment and evaluation procedures. 

5. Strategies for ensuring that the project will be dynamic, flexible, and 
responsive to the learners’ needs. 

It is important for (he staff to validate the learners’ native Ian* 
guage within the literacy framework. Staff members need to support 
the learners' continued use and development of their native language 
for improving their literacy and strengthening their family. The 
learners' literacy experiences in their native language should also be 
analyzed and considered when designing the instructional program. 
Standardized proficiency and achievement tests are available in 
various languages and may be helpful in assessing the parlicipants' 
native language proficiency and educational background. Alternative 
approaches such as a writing sample may also be useful. Depending on 
the learners' needs, family literacy projects may provide ( 1 ) opportuni- 
ties for further literacy development in their native language, (2) 
activities in which learners acquire challenging content through their 
native language, and (3) strategies for using learners' native language 
to support their children's development. A native language literacy 
program is especially helpful in giving learners successful schooling 
experiences before they have to adjust to an all-English environment. 
Many literacy projects also report that instruction in native language 
literacy facilitates the learners' acquisition of English and enhanced 
self-esteem (E. P. Garza, personal communication. May 14. 1993). 

An Expanded View of Literacy 

Any assessment of literacy-related needs should address the 
social contexts in which language and literacy are used. Literacy 
occurs everywhere in the adult learner's world: at home, in the work 
place, at school, and throughout the community. Furthermore, the very 
nature of literacy goes beyond communicating meaning through 
speaking, reading, and writing. As suggested by Eisner ( 1991 ). literacy 
can be considered “the ability to encode or decode meaning in any of 
the forms used in a culture to represent meaning" (p. 1 1). Language, 
according to Eisner, is just one form of literacy; meaning is also 
conveyed through music, dance, and visual forms. 

The importance of Eisner's expanded paradigm of literacy 
became apparent to Lincoln Unifted's staff when the results of the 
needs assessment survey (Example 1, see page 50) revealed that 
although the majority of the Cambodian participants never attended 
school in their native country and could not read or write in their native 
language (Khmer), they were experts at communicating meaning 
through other media, such as painting, music, classical dance, and 
wood carving. By recognizing the participants' lifelong achievements 
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and their unique levels of literacy, staff were able to create a program 
design that included activities to enhance the learners' self-esteem and 
build on their abilities. For instance, their program design included 
activities in which adult learners shared their areas of expertise (for 
example, native cuisine, folk dance, music, and sewing) with others. In 
developing the instructional program, it is important to build on what 
learners are already doing with literacy. A successful needs assessment 
will reveal learners’ successes with literacy as well as their needs for 
expanding their uses of literacy. 

Initial Assessment in Implementing 
a Successful Project 

This section will describe the integral role of assessment in 
successfully implementing the project and refining the objectives, 
curriculum content, and instructional activities. Initial assessment is 
important in the implementation of any project because it allows 
project staff to ( I ) reshape and refine their literacy framework, (2) 
obtain useful information about the learners during initial intake, and 
(3) establish important baseline data for assessment and evaluation. 

Refining the Literacy Framework 

A project's literacy framework should be synchronized with the 
needs, goals, past experiences, and cultural values of the participants. 
To ensure the relevance and effectiveness of the literacy framework, 
the staff can use survey results to guide them in the early stages of 
implementing the project. For example, the results of surveys can be 
used to identify curriculum content and align project goals and objec- 
tives with learners' needs, interests, and past literacy experiences. 
Administering surveys also has the effect of publicizing the availabil- 
ity of literacy classes and attracting learners to the project. 

During the initial stages of implementing the family literacy 
project at Lincoln Unified, staff visited individual apartments to 
administer a parent survey (Example 3, see pages 52-53) orally in 
Khmer to prospective participants. This needs assessment was a 
follow-up to the survey conducted with Example 1. In addition to 
providing important data about the participants and the community, the 
survey enabled staff members to develop an in-depth awareness of the 
personal and educational needs and goals of the Cambodian families. 
The parent survey contains open-ended questions designed to elicit 
comprehensive responses. Administration of the survey helped staff 
and learners begin to build a bond of mutual trust and respect that is 
essential to the success of a family literacy project. 

Another way to maintain the relevance and effectiveness of the 
literacy framework is to conduct focus groups with learners at the 
beginning of the project and at regular intervals throughout implemen- 
tation. During the focus group activities, staff can check the learners' 
own assessment of their progress and obtain their input regarding the 
relevance of the instructional program. 
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Establishing an Intake Assessment Procedure 

The second purpose of initial assessment during the implementa- 
tion of a project is to provide the staff with intake data useful for 
several purposes; meeting the needs of individual learners, developing 
appropriate curriculum content and instructional methodology, and 
placing learners in appropriate instructional contexts. During this 
phase of initial assessment, staff can collect background information 
on actual project participants, including proficiency in English and 
their native language, previous educational experiences, and length of 
residency in the United States. Instruments for collecting basic intake 
information are shown in Examples 4 and 5 (see pages 54-56). 

Because they are often the learners' first in-depth experiences 
with the project, intake procedures become part of the foundation for 
establishing a good working relationship between staff and learners. 
Therefore, intake procedures need to give learners a positive intro- 
duction to the project and provide staff with useful information for 
planning instructional activities. Staff members can help learners feel 
comfortable during intake by employing appropriate cross-cultural 
strategics and using the learners’ native languages. 

Most family literacy projects no longer rely solely on standard- 
ized tests for obtaining intake assessment data. Alternative approaches, 
such as interviews and writing samples, can provide staff with mean- 
ingful, highly personalized insight into the learner’s background. The 
interview is a versatile assessment approach for obtaining many kinds 
of information and can range in length from a five-minute informal 
exchange loan elaborate two-hour session consisting of “an array of 
questions and activities designed to engage adults in self-exploration of 
their literacy practices and abilities” (Lytle, Belzer, Schultz. & 
Vannozzi, 1989, p. 59). 

Auerbach ( 1992, pp. 1 17-18) developed an abbreviated list of 
Lytle et al.’s interview questions according to categories of informa- 
tion about prospective learners. The following are some examples from 
each category: 

Students ' background 

• Where are you from? 

• What is your native language? 

• Do you have family here? 

Employment 

• Did you work in your native country? What kind of work 
did you do? 

• Do you work here? 

• Do you do work that you are not paid for (for example, child care 
and community service)? 
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• I )id you £.0 (o school in your native country? 

• Do most people in your native country go to school? 

• Did your parents go lo school? 

Conceptions about litem . y 

• Do most people in your native country know how to read and write? 

• How is reading taught? 

Reading 

• Do you like to read? Why? Why not? 

• Do you read at home? What do you read? When do you read? 

• What languages do you read? 

Writing 

• Do you like to write? Why? Why not? 

• Do you write at home? What do you write? When do )ou w rile? 

• What language or languages do you write? 

Support systems 

• What do you do when you have trouble reading or writing 
something? 

• Does anyone help you? Who? 

• 1 X) you help anyone with reading and writing? Who? 

Auerbach emphasizes that in order to make the interviewee feel 
as comfortable as possible, the interviewer should be ready to conduct 
the interview in the native language if necessary. Furthermore, to 
create a nonihrealening atmosphere for prospective learners, inter- 
viewers should avoid taking notes during the interview. Important 
information or summary comments should be recorded after the 
interview. Interviewers may need to be trained in the use of effective 
interpersonal communication techniques so that a positive climate is 
established for the interview. Chapter V includes more information on 
the effective use of interviews. 

Lytle et al. ( 1989 ) and Auerbach (1992) suggest that, in addition 
to direct questions, interviews may include opportunities for learners 
to explore their abilities to read and write in their native language and 
in English. A reading sample might be used during the interview to 
assess the learner's reading ability. For example, an adult learner could 
be presented with a variety of reading materials in both languages: 
newspapers, story books, labels front food, greeting cards, rental 
agreements, letters from school, or prescription labels. Learners then 
could be asked to select items they would like to read while staff assess 
their reading ability. For an assessment of writing ability in their native 
language or in English, learners might be asked to produce writing 
samples as shown in Example 6 (see pages 57-58). The writing 
samples may be scored holistically because the staff is interested at this 




Assessing Success 



54 



Information obtained 
from writing samples can help 
staff members develop insight 
into learners’ initial writing 
ability and place them in 
appropriate instructional 
settings. 



point in assessing the learner's overall writing ability. Holistic scoring 
yields a single numeric value ( usually between one and six ) given to a 
writing sample as a whole. Chapter IV contains other suggestions and 
instruments for assessing reading and writing skills. 

Another practical approach to intake assessment is to use some 
lime during initial instructional activities to obtain information from 
learners that can help staff members make decisions about placement, 
cuiriculum content, and instructional activities (Auerbach, 1992). For 
example, during an introductory lesson, staff could include a w riling 
exercise in which learners are given such writing prompts as these: 
Describe a favorite place in vonr country. Describe a favorite food. 
Explain )vbyyou are happy (or not happy) about liviny in the United 
States. The responses could provide staff with valuable information 
about the learners' pres ious experiences, current needs, pd language 
abilities. 

Information obtained from writing samples can help staff mem- 
bers develop insight into learners' initial writing ability and place them 
in appropriate instructional settings. A series of at least three writing 
samples is useful for determining where to place learners in a multilevel 
program. Staff max - want to ask participants to write about selected 
topics in their native language as well as in English. The samples can he 
kept in a portfolio for comparison with learners' work later in the 
project. For learners who have not yet learned to write, stuff members 
can transcribe stories that learners tell to staff members or other learners 
in their native language or in English. 

Reading proficiency also can be assessed during regular class 
activities. Project staff can provide various reading materials in the 
learners' native language and in English, such as stories, newspapers, 
poems, recipes, and personal letters. Learners can then choose an item 
and read it to the class. For more in-depth assessment, staff members 
can use a similar procedure with an individual learner while other staff 
members work on different activities with die rest of the class. 

Collecting Baseline Data 

The third purpose for using initial assessment procedures in the 
implementation stage of a project is to collect baseline data that estab- 
lish a stalling point regarding a learner's literacy development, 
parenting experience, knowledge of community resources, self-esteem, 
confidence, interaction with children, and other outcomes that the staff 
would like to assess. Some family literacy projects use standardized 
tests to establish baseline data. It such formal assessments are used, 
learners may need a few days to feel comfortable about being in the 
project before they are tested. 

Alternative approaches such as surveys, interviews, and reading 
and writing samples, are especially useful for establishing baseline data 
because lliev can be designed according to the objectives and curricu- 
lum of the project. Samples of instruments that can be used as pre- and 
posttests are shown in Examples ”7 and 8 (see pages >9-6 1 i. These 
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Sensitivity to and 
awareness of the needs, 
literacy feyels, goals, and 



attitudes of students, whether 
they are adult learners, 
adolescents, or children, are 
essential if learning am; 
persona! growth are to occur. 



instruments can be administered to learners in English as well as in 
their native languages. 

Summary 

Assessment and evaluation are not procedures tagged onto the 
responsibilities of project staff members simply to satisfy the interests 
of a funding agency or other stakeholders. Rather, they are important 
strategies for identifying learners' needs and abilities at the initial 
stages of a family literacy project and for ensuring the successful 
planning and implementation of the project. Sensitivity to and aware- 
ness of the needs, literacy levels, goals, and attitudes of students, 
whether they are adult learners, adolescents, or children, are essential 
if learning and personal growth are to occur. 

The project evaluator is an important contributor to the assess- 
ment process. The evaluator needs to work closely with project staff to 
develop instruments and procedures for initial assessment. The evalua- 
tor can be helpful in designing instruments that are valid and reliable 
and are linked to project objectives. 

Chapter IV ot this document provides a detailed description of 
instruments and procedures for assessing ongoing progress, and 
Chapter V explains how the results of initial and ongoing assessment 
procedures can be collected, analyzed, used, and reported for various 
purposes. Annual evaluation reports that combine the results of formal 
and informal, qualitative and quantitative, and authentic and standard- 
ized measures will provide a comprehensive, credible picture of the 
project-level success achieved by all participants in a family literacy 
project. 
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Example© 

NEEDS ASSESSMENT SURVEY 



Name 



Telephone ( 



Address 



Native Language 



1. How many adults live in your apartment? 

2. How many children in your apartment attend Lincoln Unified School District? 
(Identify individual schools by name) 

3. How many adults currently attend GAIN program English classes? 

4. Where do they attend classes? (e.g., Delta College, adult school) 

A If there are adults living in this apartment who are not attending GAIN English 
classes, would they attend GAIN English classes if they were held in your 
apartment complex? 

6. How many men would go? 

7, How many women would go? 

8. Would these adults agree to attend class four hours a day. five days a week? 

9, Would they need to have someone outside the family take care of the' 1 r 
children during class? 

10. How many children need child care? 

1 1. Should class he held in the morning or in the afternoon? 

12. Is it a good idea for us to try to gel our classes approved for GAIN, or should 
we have our own literacy program? 

i 3. If we cannot gel our English classes approved for GAIN, would you attend 
anyway? 

14. How many hours per day should classes he held? 

15. How many days per week should classes be held? 

16. What would you like to learn 0 

a. English d. How the schools operate 

h. How to get a job e. How to become a citizen 

e. How to be a good parent 
1 7. What is your level of English language skills? 

a. Oral skills: ( 1 ) Low (2) Medium (3) High 

b. Reading and writing skills: ( 1 ) Low (2) Medium (3)Miuh 

18. How long have you been in the 1 mited Stales’, 7 

19. How many children in your apartment arc participating in LiSEA Title VII programs? 



L 

s 

3. 

4. 

5. 
f). 

7. 

8 . 

9. 

10 . 
II. 



13. 

14. 

15 . 

16. 



17a. 

17b. 

18 . 

19 . 
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Example© 

OMS-FELP NEEDS ASSESSMENT FOR PARTICIPANTS 



English 

Oral Skills: Where do you wan! to use better English? 

grocery store children's school 

department store telephone 

doctor's oi Tice library 

drugstore immigration office 

hank welfare office 

post office _ neiehbor/friend's home 



Reading Skills: What do you want to recu 1 better? 
school reports sig 



labels 

forms 

driver's manual 
income tax forms 



newspapers 

schedules thus, train) 

advertisements 



Writing Skills: What do you want to write better 7 

notes to children's schools fill out forms 

schedules for children business letters 



social security office 
apartment office (managei 
restaurant 

flea/fa rmcr's market 

temple/chureh 

other 



school letters 
business letters 
checks (from jobs 
job applications 
other 



checks 

other 



Parenting: What do you think is important to learn hi a parenting el ass? 

how to discipline children 

how to talk/share feelings with children 

how to help children fed good about themselves and their culture 

how to help children adjust at school 

good nutrition 

health (nutrition, personal hygiene, healthy habits) 

making the home and play areas safe 

monitoring home activities, including the use of television 

other 
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PARENT SURVEY 



Name j Telephone 



Address 



Native language l Native country 

i 

Can you read in your native language? j Can you write in your native language? 

T i | 

Year of arrival in U.S. 1 Single mother ; Single father j Married 




1. Are you presently attending a GAIN class? Yes No Where? 

2. What time is best for you to attend FELP classes? . . morning afternoon . evenings 

3. What days are best for you to attend class? 

__ Monday . Tuesday . Wednesday _ . Thursday . Friday 

4. What do you need to help you attend classes? ~ Child care ’ Translator 

Friend to come with you . Other 

5. What would you like to learn? 

. Apartment management (how to work with the apartment manager, get something repaired, ask about 
rent, look for an apartment) 

Citizenship (how to become a U.S. citizen) 

Communication with legal authorities (attorneys, police, legal support. legal aid. neighborhood 
watch) 

Education (registration, how to notify the school of illness, talk to the teacher/principal, visit your 
child's school, understand grades and progress reports) 
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Parent Survey (continued) 



English as a second language (oral communication, reading, and writing) 

Health (immunizations, hygiene, medical and dental check-ups. how to talk to the doctor and dentist, 
fill out medical forms) 

Mathematics (money, banking, numbers, addition, subtraction, multiplication, division) 

Nutrition (common foods in the United States, healthful eating) 

Parenting/child rearing (developmental stages, suggestions on how to help your child at home, how to 
communicate effectively with your child) 

Safety (emergencies, car seats, fire prevention, water safety, first aid) 

Other 



6. What kind of help do your children need? 
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Name tLasn 


( liisU 


(Middle) 


Address <Siroi 


if 'il\ ) 


< Stale) fZII J C’odiM 


Telephone number ( ) 


. Birth date 


Male Female 


Native country 


! Date of arrival in U.S. 




Did you go to school in your country? 


Yes No How long? 




Do you read in your native language? 


Limited Fluent 




Do you write in your native language? 


Limited Fluent 




What other languages do you speak? 







What other languages do you read? 



What other languages do you write? 



Have you taken English classes before? Yes No How long? 

Where? 

Are you: Married Divorced Widowed 

Single Separated 

Name of hushand/wife? 

Number of children Ages 

Health problems 



Social Security number 
Alien registration number 

MM*' m*tm w— i mma — — w — ■ . a 

Office use only — Please do not write below this line. 

Date application received: 

Dale of test/s: 



Family English Literacy Project • Lincoln Unified School District • Stockton, California 



Assessing Success 



62 





Example © 

REGISTRATION TO RM 

(1 o be completed by stuff about family members participating in the project.) 



Family name 



Father’s name 



Mother's nanx 




Address 



Telephone ( 



What country do you come from? 



How long have you been in the United States? 

What level of education 
did your father complete? 

Did your children go to school in your country? 

Father's work 



How long have you been in Solana Beach? 

What level of education 
did your mother complete? 



Mother's work 



Are you enrolled in any other BSD class? Yes No 
Do you have books at homo? Yes No 

How many books do you have at home? 1 — 10 1 1-20 

Are the) in English or Spanish? English Spanish 

Do you have a library card? Yes No 
Do \ou have a tclevbion in \our home? Yes No 



21-39 More than 40 
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Registration Form (continued) 



Do you watch television programs in English or Spanish'? English Spanish Both 

Do you listen to radio programs in English or Spanish? English Spanish Both 

Do you read newspapers in English or Spanish? English Spanish Both 

Do you correspond with your family in Mexico? Yes No 
Does the father: 



Understand spoken English? Yes 


No 


Speak English? 


Yes 


No 




Read English? 


' Yes 


. No 




Write English? 


? Yes 


No 




Read Spanish? 


Yes 


No 




Write Spanish? 


Yes 


No 




Does the mother: 








Understand spoken Engl 


iish? Yes 


No 


Speak English? 


■; Yes 


No 




Read English? 


\ Yes 


No 




Write English? 


Yes 


No 




Read Spanish? 


Yes 


... No 




W, lie Spanish? 


Yes 


No 




Do the children: 








Understand spoken English? Yes 


No 


Speak English? 


; Yes 


No 




Read English? 


' Yes 


No 




Write English? 


Yes 


No 




Read Spanish? 


Yes 


. No 




Write Spanish? 


Yes 


No 




What problems have 


you encountered in tin 


: United States that you would like to discuss in class? 



Family English Literacy Project • Solana Beach Elementary School District * Solana Beach, California 




Assessing Success 



64 



(I'irhli 



( A tic i 



Name ii.asu 
Write numhers 1-27 

1 



Write the alphabet. 

A 



2 7 



_ Z 

Read the sentences and circle the correct word in the box. 

1 . He j go at went ! to work. 

2 . She j am is are : thirtv-llvc. 

3. How old , am is are you? 

4. 1 go going went , to buy food yesterday. 

5. We | came come came from to the United States five years ago. 

6. They! has have got , five children. 

7. 1 . has had have been studying English for six months. 

Family English Literacy Project • Lincoln Unified School District • Stockton, California 
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PRIMARY LANGUAGE WRITING SAMPLE 




Write about the picture in your native language. 






Circle the foods tha? are healthy snacks. 




carrots orange milk potato chips cookies 



Circle the correct answer. 




Make an X on the foods that are riot healthy snacks. 




Adapted from Holt. G. D.\1S88). Parenting curriculum for language minority parents. Sacramento California State University, Cross 
Cultural Resource Center. 
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Example © 

PRE/POST TEST: THE U.S. SCHOOL SYSTEM 



Circle the one correct answer for each of the questions below. 

1. The report a school sends home to tell about a student’s progress in school is culled a 

A. Cinch nonce 
13. Emergency card 

C. Report card 

D. Absence report 

2. The top person in an elementary school is called a 

A. Librarian 
13. Principal 
C. Teacher 
I). Vice-principal 

3. High school ends at grade 

A. Six 

B. One 

C. Nine 
F), Twelve 

4. A student checks out hooks from the 

A. Cafeteria 

B. Playground 
(\ Office 

1). Library 

5. When a child is too sick Lo go to school, he or she should 

A. Stay home and play outside 

B. Rest inside the house 

C. Watch TV all day 

D. None of these 

6. What tilings arc needed for a child when registering for school? 

A. Birth certificate 

B. Shot records 

C. Physical examination 

D. All of the above 

Answer the following questions true (yes) or false (no). 

7. Children need more sleep than adults 

8. A parent cannot help a child in school if the parent does not speak English. 

9. Schools do not want the parents to help their children with school work. 

10. Teenagers go to high school. 
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Pre/Post Test: The U.S. School System (continued) 

If you have a child in school, answer the following questions. 

12. What grades are your children in? 



13. What schools do your children attend? 



14. Where are your children's schools located? 



15. What are the names of your children's teachers? 
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CHAPTER IV 

ASSESSING 
PROGRESS: 
ARE WE 

PROGRESSING? 




Heide Spruck Wrigley 
Southport Institute for Policy Analysis 
Washington, DC 



To judge how well a family literacy project* is working on a day-to-day 
basis* staff members need to take the educational pulse of the program 
periodically and assess learners’ progress. Progress assessment can help staff 
determine the extent to which their instructional efforts lead to beneficial 
results and have positive effects on the learners' lives. Assessments also give 
participants a sense of accomplishment by showing them that they are 
making headway toward attaining their goals (Wrigley, 1995). To be effec- 
tive, progress assessments need to include several components: 

1 . Baseline data that show what participants know and what they can do. 

2. Indicators of progress or descriptors of what counts as success. 

3. Methods to gather information on a continuous basis. 

4. Plans for s\ nthesizing and analyzing the data that are collected. 



’ See the ( ilossaiy on pages |2 ( ) 131 <>l ihK publication I'm Udmilions ol Halu t:vii hc\ terms that 
appear in this chaplet 



This chapter shows how staff can develop progress indicators 
based on the objectives of their project (see Chapters I and II for more 
on objectives). Six sets of instructional activities are presented, each of 
which focuses on particular learner outcomes and provides examples 
of alternative approaches for assessing the learners' progress toward 
the outcomes. These activities include the following: ( i) helping 
learners and staff get acquainted, (2) linking project activities with 
family life, (3) reading to do. (4) writing to do, (5) building self-esteem 
and taking literacy beyond the classroom walls, and (6) developing 
literacy through language experience. 

This chapter illustrates how learners' performances in these 
activities can be assessed with a variety of approaches such as observa- 
tion measures , interviews, and performance samples. The examples of 
alternative assessment instruments presented in this chapter are not 
intended as ready-to-use assessment tools. Staff members should adapt 
the instruments and procedures based on the unique characteristics and 
objectives of their project. Before presenting the activities and instru- 
ments, a rationale will be provided for using alternative approaches to 
assess learners 1 ongoing progress. 



Alternative assessments 
can focus on the way learners 
use language and literacy as 
opposed to how much they 
know about grammar, phonics, 
or spelling. 



Characteristics of Alternative Approaches 
to Assessing Ongoing Progress 

Conventional assessments often use pre- and post-tests to com- 
pare how much learners know before and after an instructional unit or 
employ standardized measures that may not reflect the form and 
content of the program. Alternative assessments can go one step 
further. Rather than focusing only on knowledge and skills, they seek 
to show changes in learners' attitudes, social interactions, and commu- 
nication patterns that may be attributed to their participation in the 
program. In contrast to many standardized assessments . alternative 
approaches can link instructional content with assessment. In addition, 
alternative assessments are often participatory, involving learners as 
partners in the assessment process. 

Linking Curriculum with Assessment 

Alternative assessments can focus on the way learners use 
language and literacy as opposed to how much they know about 
grammar, phonics, or spelling. For example, a traditional test might 
assess whether students know irregular verb tenses, can find words that 
rhyme with “cat," label the parts of the body, or answer comprehension 
questions about a paragraph. An alternative assessment, on the other 
hand, might seek to examine how well a learner can explain a problem, 
read a report card, or write a note to a neighbor. Alternative assess- 
ments also can be used to report on the role that literacy plays in 
learners' lives and to document the life changes that lake place as 
learners become involved in family literacy. 

Alternative assessments are particularly useful with a language 
and literacy curriculum that emphasizes meaning over form and 
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communication over structure. Such assessments are also effective for 
learner-centered approaches to teaching English-as-a-second-lan - 
guage(ESL) literacy. One recent research study on successful pro- 
grams and promising practices defined a learner-centered approach to 
ESL literacy as “supporting language-minority adults with little 
English and little formal education in their efforts to understand and 
use English in its many forms (oral and written, including prose, 
document, and quantitative literacy), in a variety of contexts (family, 
community, school, work), so that they can reach their fullest potential 
and achieve their own goals (personal, professional, academic)' 1 
(Wrigley & Guth, 1992, p. 7). 

Because a learner-centered curriculum tries to be responsive to 
the participants’ changing needs and goals, learner-centered assess- 
ment also needs to be dvnamic and flexible. Such assessment should 
be capable of capturing incidental learning and unforeseen problems 
and of demonstrating the extent to which program objectives have 
been met. A collaborative effort among teaching staff, learners, and 
those evaluating the project helps to ensure effective planning and 
implementation of alternative assessment (see Chapter II fora detailed 
description of the planning process). 

The next section is the first of six sets of instructional activities 
that illustrate a variety of alternative approaches to assessing ongoing 
progress. Linking instructional activities with assessment provides a 
multifaceted view of the learners' involvement and progress in a 
project. 



During the early stages 
of a project, it is important for 
learners to have opportunities 
to discuss the role that 
literacy plays in their lives. 



Helping Learners and Staff Get Acquainted 

During the early stages of a project, it is important for learners to 
have opportunities to discuss the role that literacy plays in their lives. 
Through a series of exploratory activities, learners may identify their 
goals for literacy, describe the ways they use literacy at home and in 
the community, explain (heir hopes and dreams for their children, and 
talk about their interests in acquiring literacy in their native language 
and their goais for learning English. This section presents activities 
and assessment procedures that can be used at the beginning of the 
instructional program. These activities also provide opportunities for 
teachers and learners to get to know each other during the initial stages 
of the project. In addition, they present a chance to discuss how the 
program can meet learners’ needs and help families move toward their 
goals. (Chapter HI provides initial assessment strategies for obtaining 
information for planning and developing instructional objectives and 
curriculum content.) 

Surveys and questionnaires can be used just before classes begin 
to assess the learners’ needs. After instruction is underway, additional 
information can be gathered effectively in group sessions in which 
learners discuss the roles that English and literacy play in their daily 
lives. These sessions may result in literacy inventories that reflect 
when, where, and for what purposes learners engage in certain types of 
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To develop useful 
learner-centered assessments, 
staff members need to gain 
and maintain the trust of the 
learners so that they feel free 
to share their experiences, 
ideas, and aspirations with 
their peers. 



reading and writing; for example, is reading religious material more 
important to them than reading the newspaper? Activities that invite 
learners to talk about how they use literacy and what they want to be 
able to do with literacy should occur at regular intervals throughout the 
project. Such activities will help learners express their goals more 
clearly and identify ways in which literacy can enrich their lives. To 
develop useful learner-centered assessments, staff members need to 
gain and maintain the trust of the learners so that they feel free to share 
their experiences, ideas, and aspirations with their peers. 

Small-group and pair activities are especially helpful in allowing 
staff to ( 1 ) note how learners interact with each other, (2) observe the 
learning strategies they use, (3) gain insights into the learners' commu- 
nication skills and literacy practices (in English and in their native 
language), and (4) recognize the skills and abilities that learners bring 
to class. 

What to Do 

To find out more about the learners and help them get to know 
one another, staff members can use ( I ) grids in which participants 
match information about a classmate with the person's name: for 
example, "Who has two children and comes from Vietnam?"; (2) maps 
and pictures from magazines that allow learners to talk about their 
native countries; (3) photographs brought from home that enable 
learners to talk about their families; and (4) language experience 
stories in which learners tell about a significant event such as, "My 
first day in the United States." 

Learners also can get acquainted by interviewing each other. 
Gillespie ( 1990) suggests that teachers cut pictures in half, ask learners 
to choose half a picture out of a box, find the person who has the other 
half, and interview that person. Interview topics could include ques- 
tions about the learners: 

1. Interests, likes, and dislikes. 

2. Background knowledge, skills, and occupation. 

3. Reasons for enrolling in the project. 

4. Family life and parent-child activities. 

5. Experiences with schooling in their native country and the United States. 

Follow-up activities can address such questions as, "When and 
where do you use English and why?"; “When do you use your native 
language?"; and "What are your educational goals for yourself and for 
your children?" Learners also can be asked about the role of literacy 
brokers in the community— individuals who act as cultural guides to 
IJ.S. society and help newcomers negotiate paths through the maze of 
ITS. bureaucracies and other obstacles (Weinstcin-Shr, 1990). Of 
particular interest for family literacy might be the role that children 
play in mediating language for their parents and grandparents. Staff 
can use such investigations to create in-class discussions about cross- 
cultural differences in communication, schooling, or home health care. 
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Collaboration among 
participants helps create a 
community of learners and 
allows staff to assess learners 
along various dimensions of 
literacy. 




These discussions could then lead to instructional activities that provide 
opportunities for sharing feelings and ideas and for acquiring other 
knowledge and information. 

Assessment Strategies 

Activities in which participants interact, learn together, and teach 
each other provide staff with opportunities to assess what learners 
know, what role literacy plays in their lives, when and where they use 
English and their native language, and what educational values they 
hold. Collaboration among participants helps create a community of 
learners and allows staff to assess learners along various dimensions of 
literacy: 

1 . Levels of self-esteem and confidence in interacting with others. 

2. Ability to understand spoken and written English. 

3. Abiliiy to express ideas orally and in writing. 

4. Patterns of native language use aL home and in the community. 

3. Knowledge about cross-cultural differences in institutions, services, and 
life-skill areas. 

6. Levels of independence and degree of transfer of skills and knowledg v 
beyond the classroom, including changes in behavior desired by 
participants. 

Staff members who are uncertain about which of these dimensions 
to focus on initially can design an observation measure for each learner 
that captures information in each of the areas until categories worth 
pursuing begin to emerge. Or staff can create a simplified version that 
would focus on fewer areas. Example 1 (see page 82) is an observation 
measure that can be used to obtain basic information about the learners. 
The information obtained can be used for the following: 

L To generate baseline data for monitoring learners' progress over time. 

2. To provide information for staff members to share regarding individual 
learners and to detect common patterns in the learners' responses to 
classroom activities. 

3. To create classroom activities that address learners' needs. 



Staff members can use questions such as the following to guide 




their discussion and to develop descriptors for various levels of 
progress: 

1 . What differences do we .see between advanced and lower level learners in 
reading, writing, speaking, and listening abilities? 

2. What evidence of coping skills do we observe? What do we know about the 
strategies that learners use to solve problems and communicate in English 
even though they have only minimal English and are just barely literate? 

3. flow much do indiv idual learners already know about laws, regulations, 
and institutions that affect their community (for example, schools, health 
care providers, and social service agencies)? 

i~-'! Um V AVAIL ADLE 




Chapter IV 



mjfjP By developing 
descriptors of learners’ 
behavior at various stages of 
development, staff are able to 
identify changes in learners’ 
performance over the course 
of the project. 



Initial program activities 
need to encourage participants 
to learn from and teach each 
ot&tar by observing and doing. 



4. What evidence do we have that learners are applying in their personal lives 
what we are leaching in ihe project ? 

5. What are the characteristics displayed by independent learners? How are 
such learners different from those who are overly dependent on their peers 
or the teacher? 

By developing descriptors of learners' behavior at various stages 
of developments staff are able to identify changes in learners' perfor- 
mance over the course of the project. Staff members can continue to 
add meaningful information to learner profiles as the project is imple- 
mented. When assessment and instructional activities work in tandem, 
learners' needs and their progress are illuminated, enabling staff to 
gain clearer insights into how the project can be tailored more appro- 
priately to the learners. 

Figure 1 contains comments that staff members might write 
about learners in the learner profile (see also Example l , page 82). 

Linking Project Activities with Family Life 

This section shows examples of how instructional activities, goal 
setting, and assessment can be linked with family life. First, learners 
work together to create some kind of joint project. Next, parents talk 
about the kinds of activities they do with their children at home and 
discuss how these activities can support children in their own learning. 
Parents then decide what they would like to do with their children 
during the next class sessions. Finally, parents talk about the time they 
spent with their children and what their experiences have meant 
to them. 

What to Do 

Initial program activities need to encourage participants to learn 
from and teach each other by observing and doing. For example, staff 
members, parents, and children can work and play together to create 
collages, holiday greetings, announcements, or birthdry cards. Staff 
might start by showing some interesting examples and by demonstrat- 
ing a particular technique for creating the materials. While they work 
with paper and scissors, participants learn the English names for 
colors, shapes, and actions such as cutting, folding, and tearing. 

In subsequent activities, participants share information, describe 
their materials, or teach each other a particular technique. Similar 
activities can include making paper airplanes, origami (paper folding), 
or party decorations. ( For an example of a lesson on making and 
playing traditional instruments, see Quintero & Huerta-Macfas. 1990.) 
Next, parents can discuss some of the activities they do with their 
children at home and talk about the kinds of reading and writing they 
do in their native language or in English. 
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Examples of Comments for 
the Learner Profile/Observation Card 

Communication skills 
Listening comprehension 

Felicia still looks a bit overwhelmed and confused when we discuss 
things or when I give directions for a particular task. She seems to 
understand more when I talk to her directly, especially when I ask her 
about her family. She does not seem to understand very well when we 
venture into topics that go beyond her family, her home, or the town 
that she came from (Guadalajara). 

Reading 

Felicia seems eager to read. Whenever ! hand out the language 
experience stories that the class has written, she immediately gets 
started. However, she seems to get stuck easily whenever she 
encounters an unfamiliar word or a difficult phrase. She is able to 
answer simple comprehension questions about a story we have talked 
about in class but has difficulty with new readings. She seems quite 
ready to link her own experience to other people’s stories and her 
responses to the story focus on recounting the facts presented. I am 
hoping she will talk about herself more and see the connections 
between her own life and the experiences of other students in the 
class. Felicia picks up magazines during our free reading periods and 
leafs through them but she does not seem engaged in any of the texts. 

Coping strategies and problem solving 

Although Vuth scored fairly low on our standardized placement lest, 
he seems to cope pretty well in class. He is able to figure out what to 
do when I assign a task by watching other students or asking an 
occasional question. He is willing to put pen to paper when I ask the 
class to write (although his written English is very difficult to 
understand), and he eagerly looks through the books and magazines 
we have in the classroom. Although he has a friendly smile, his tone is 
still rough at times, especially when he asks for something. ("Teacher, 
give me book” is a typical request.) 

Content knowledge and understanding 
of institutions, services , and life skills 

Marta seems interested in how the community works, but she has very 
little knowledge of the services that are available to her (she did not 
know she could get free prenatal care at the clinic). A friend brought 
her to class, and she has only been in California for two months. She 
does not know the name of the school her children attend or the 
names of their teachers. She does know that 911 is for emergencies 
but thought that all she has to do is dial and the police will come right 
away to the right address (she was surprised to hear that she would 
have to explain the nature of the problem). She did not know that she 
could ask the operator for help in Spanish. 
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Activities that involve 
parents and children working 
together offer staff a starting 
point for discussing the social 
interaction among family 
members and for understanding 
the literacy practices used in 
the home. 



Assessment Strategies 

Activities that involve parents and children working together 
offer staff a starting point for discussing the social interaction among 
family members and for understanding the literacy practices used in 
the home. To learn more about the participants’ families, staff can 
develop questionnaires in which parents, working individually, 
in pairs, or in groups, list the activities they do at home with 
their children. 

Example 2 (see page 83) is an illustration of a questionnaire for 
generating a list of activities that parents and chi Idrcn do together. The 
list generated by the learners can become baseline data for document- 
ing changes over time in parent-child interactions (see Example 3, 
page 84). The assessments shown in Examples 2 and 3 could be 
conducted in the learners’ native language or in English. 

In ensuing activities, staff members can provide information to 
parents about learning strategies used by children and present ex- 
amples of how parents can support their children's learning at home. 
For example, parents can give children opportunities to observe others, 
learn new things with siblings, ask questions, or discuss cause-and- 
effect relationships. Using a questionnaire such as the one shown in 
Example 4 (see page 85), parents can discuss activities they consider 
useful for helping their children. 

Next, parents can identify the kinds of activities they plan to 
undertake with their children and the frequency with which they might 
conduct each activity (see Example 5, page 86). These activities could 
become part of an individualized plan that helps learners document 
progress toward meeting (heir own goals and the objectives of the 
project. Parents could use a questionnaire such as Example 6 (see page 
87) for reporting the activities they try with their children and for 
determining to what extent they have met the goals in their plan. 

Examples 4-6 can be adapted to meet the specific purposes of a 
family literacy project. For example, the complexity of the English 
vocabulary and structures used in the instruments could be tailored to 
the learners’ ability. In addition, the instruments could be translated 
and administered in the learners’ native language. Some projects also 
might have staff members or learners use the instruments for indi- 
vidual interviews in English or the native language. 

Reading to Do 

Beginning learners need opportunities to deal with different 
kinds of texts and cope with a variety of forms of English. Storytelling 
and language experience activities show how sentences are connected. 
To focus on one word or sentence at a time, learners can read signs, 
symbols, and instructions that appear in their environment every day. 

The activities in this section are designed to help participants 
develop their receptive skills of listening and reading. The focus of the 
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activities is on introducing beginning learners to reading through 
environmental print (signs and symbols in the community) and impera- 
tive print (messages that tell citizens what to do and not to do). An 
observation measure is presented for assessing listening and reading 
abilities. 

What to Do 

Field trips can be used to give adult learners opportunities to 
operate a variety of machines such as those for making photocopies or 
obtaining soft drinks and change. Participants also can learn how to use 
a public telephone or operate an automated teller machine. 

Working in pairs or small groups, learners discuss the instruc- 
tions for the machines and practice what is required to operate them. 
Learners might use a Polaroid camera for taking pictures of instruc- 
tions they find hard to understand. During a field trip, they also can 
write examples of imperative print, such as Exit, Stop, and No Turn 
on Red. Staff members could select several sets of the instructions for 
additional practice in class. 

During classroom activities, learners can identify the machines 
they found particularly useful, interesting, or difficult to operate. They 
might decide which instructions they would like to study further in 
literacy activities such as reading simple imperatives, matching 
instructions with appropriate machines, or putting instructions in 
sequence. For example, the sequence of instructions for using a public 
telephone would be (1) lift receiver, (2) listen for the dial tone, (3) 
deposit coins, and (4) dial the number. 

Follow up . Learners can develop language abilities related to 
operating other machines such as an audiotape recorder, camera, 
videocassette recorder, three-hole punch, subway ticket machine, or 
computer. As learners become more proficient, they might work with 
longer and more complex instructions for such tasks as assembling 
simple toys, making crafts, following recipes, or completing job 
application forms. 

Assessment Strategies 

As learners try to figure out how things work by reading or 
listening to instruct Lons, staff members can observe their reactions and 
assess the learners' reading and listening skills. Similar activities 
should be repeated throughout the instructional cycle to determine 
whether reading and listening are becoming easier and to compare 
learners' skills at the beginning of the course with their abilities toward 
the end. Instructions can be given orally to test listening comprehen- 
sion or in writing to assess reading skills. In both cases, the instructions 
used for assessment should be somewhat familiar to learners so they do 
not become overwhelmed or discouraged by the task. Example 7 (see 
page 88) is an illustration of an observation measure for assessing 
listening and reading abilities. 
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To develop a consensus on the meanings of the descriptors used 
(With ease. With some support, etc.) and to <. tablisli the reliability 
of the observation measure, staff members should use the instrument 
with adult learners at similar levels of proficiency under a variety of 
circumstances. With practice, learners can develop their own under- 
standing of the meanings of the descriplors and begin to evaluate their 
own performance. At more advanced stages, they can evaluate their 
peers and give constructive feedback al out their performance. 

Writing to Do 

Writing instructions for others to follow provides learners with 
the opportunity to communicate ideas to an audience. Because instruc- 
tions require clear language and a logical sequence, they represent an 
effective way for learners to write down what they know and to ensure 
that their readers understand. 

This section presents cooperative activities in which participants 
teach and learn from each other by talking about a particular skill or 
ability they possess. The activities work well in a multilevel class, 
because they give less proficient learners a chance to develop knowl- 
edge by observing and responding nonverbally while more advanced 
learners enjoy the special challenge of teaching others. An observation 
measure is presented for assessing learners' speaking and writing 
abilities. 




In cooperative activities, 



participants realize that each 



member of the group can be a 
teacher and a learner. 



What to Do 

In cooperative activities, participants realize that each member of 
the group can be a teacher and a learner. To introduce the activities, 
staff members can ask parents what they teach their children and what 
their children teach them. It may be interesting for adult learners to talk 
about what they learned from their parents. Learners might aiso 
discuss how they help their neighbors and friends. These activities 
allow staff members to identify skills participants have, such as 
repairing a flat tire, preparing a home remedy for a sick child, fixing a 
broken window, getting rid of stains, or braiding long hair. 

During these activities, staff members can validate knowledge 
learners possess such as knowing how to find a bilingual doctor, where 
to get free legal advice, how to talk to a social worker, and where to 
find a video store that stocks films in the learners' native language. 
Staff members might make a list of skills represented in the group and 
ask learners to decide which ones they would like to develop further. 
Staff can also discuss the possibility of bringing in outside speakers 
from legal aid and public health agencies, the Red Cross, or the 
community relations unit of the police department. Presenters can talk 
about topics such as fighting an eviction notice, practicing safe sex, 
gi\ ing blood, or protecting a home against burglaries. 




Assessing Success 



79 




Staff members can ask participants to choose a particular task that 
interests them such as making tamales or a fruit salad, sorting coupons 
to use at the grocery store, demonstrating first aid, playing the kazoo, 
making popcorn, putting up a bulletin board, or taking Polaroid 
pictures. Group members might work together to write instructions for 
others to follow. After discussing and demonstrating the instructions in 
small groups, individual learners can demonstrate the task to the class 
using both verbal and nonverbal cues to illustrate their points. To 
evaluate the learners' speaking and writing abilities related to giving 
instructions, staff members can use an observation measure similar to 
the one shown in Example 8 (see page 89). 



Assessment Strategies 




In preparation for a 



The observation measure shown in Example 8 demonstrates how 
an instrument can be developed for specific purposes in a project. In 
this case, the instrument was designed to measure learners' progress in 
improving their speaking and writing abilities related to giving instruc- 
tions for particular tasks. 

Another assessment strategy would be to put each learner’s 
written instructions in a portfolio for comparison with future writing 
samples. Copies of the instructions could be given to other learners to 
try to follow. Staff might use selected sets of instructions for subse- 
quent literacy activities such as fill-in-the-blank or cloze exercises, 
completing sentences that require longer phrases, unscrambling words 
in sentences, sequencing instructions, and matching instructional steps 
with pictures. 

For intermediate learners, the types of activities described here 
can be extended to teach functional literacy skills such as filling out 
school registration forms, understanding immunization records, and 
telephoning the school for information. (For other examples of literacy 
activities for intermediate students, see Holt's Parenting Curriculum 
for Language Minority Parents: Teacher's Activities Guide, 1988.) 

Staff members also can design writing activities for the whole 
class. For example, in preparation for a party or a holiday. learners 



party or a holiday, learners might work together to prepare a meal, provide entertainment, explain 



might work together to prepare 




the significance of the day, write invitations, and make decorations for 
the room. The activities could then be written up, either as work plans, 
narratives, or items for a class newsletter. The written work can 
become the basis for further language and literacy development. 

Building Self-Esteem and Taking Literacy 
Beyond the Classroom Walls 

The alternative assessment instruments discussed thus far focus 
on language use; that is, they seek to assess how well learners can 
perform specific language-related tasks. Although these instruments 
can document changes in language abilities, they do not provide 
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and use others as resources 



make significantly greater 
gains in learning than do 
students who work alone. 



information on affective factors such as self-confidence, self-esteem, 
or motivation. Nor do the instruments provide data that would indicate 
how well learners work together or to what extent they transfer knowl- 
edge gained in the classroom to their lives beyond the school. 

Affective factors deserve the attention of staff members because 
of the importance of self-confidence and self-esteem in influencing not 
only the learners’ language development but also their general level of 
success in adapting to the cultural challenges they face at home, in the 
community, and at work. Research has shown that learners who 
interact with and use others as resources make significantly greater 
gains in learning than do students who work alone (Kagan, 1986; 
McGroarty, 1993). Similarly, studies in cognition demonstrate that 
language and literacy development is increased if learners can link 
classroom knowledge to real life contexts (Anderson. 1988; Anderson 
& Pearson, 1988; Smith, 1975). 

What to Do 

Project staff can use a variety of activities to help learners de- 
velop self-esteem and to improve their interaction with others. Activi- 
ties should enable learners to demonstrate the knowledge, insights, and 
abilities they have developed in responding to real-life challenges. The 
following are activities that staff might try in their project, or they may 
want to design their own. 

1. Case studies in which learners are presented with a familiar problem and 
asked to brainstorm possible solutions. For example, a case might involve 
a daughter who wants to play on the volleyball team, but the mother, a 
single parent, is worried about her daughter's being away from home for 
practice and out-of-town games. 

2. Situations that illustrate a significant economic, social, or cultural issue, 
such as lack of affordable housing; street crime; and racial, ethnic, and 
language-related discrimination. The situations, sometimes called codes, 
can be presented through short readings, dialogues, and pictures. They can 
provide starting points for the learners to discuss their concerns and to 
brainstorm solutions. Codes are based on problem-posing pedagogy 
developed by Paulo Freire. (See Wallerstein, 1983, for similar activities.) 

3. Simulations and role plays in which learners play themselves in situations 
such as performing a cultural dance, participating in a parent-teacher 
conference, or dealing with uncooperative landlords. 

4. Projects such as developing a class biography, producing a resource guide 
for newcomers to the community, organizing activities that help their 
children learn their native language, or producing dictionaries and 
pictionaries in English and the learners' native language. 

5. Lists and charts of individual accomplishments that are posted on class- 
room walls. Learners use their own words to describe achievements 
related to language, literacy, and learning that they are proud of. 

0. Language experience stories in which learners tell of the struggles in their 
lives and celebrate their successes in becoming biliterate and bicultuiui. 
(For examples of learner-generated stories, see Weinstein-Shr, 1992.) 
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views in writing. 



Assessment Strategies 

The observation measure shown in Example 9 (see pages 90-9 1 ) 
can be used to (1 ) record changes in learners’ confidence, (2) document 
levels of participation in pair or group activities, and (3) show transfer 
' >f skills to actual life experiences. By recording information about the 
:iree affective areas in one chart, staff members can analyze relation- 
ships among the domains. For example, the data might indicate that as 
learners become more confident, they participate more freely and 
transfer the knowledge gained in class more readily to real life. Results 
of the measure might also illustrate that levels of self-confidence are 
related to levels of participation, as in the case of learners who collabo- 
rate more freely as they become more self-confident. When interpret- 
ing the results, staff members need to be aware that factors outside the 
project also may influence the learners’ level of self-confidence. 

Developing Literacy Through Language Experience 

One of the most challenging tasks faced by literacy project staff is 
introducing nonreaders to literacy. Adults who are developing literacy 
in English as well as in their own language need a great deal of time 
and special support so they can comprehend printed materials and 
express their views in writing. This section gives examples of activities 
designed to facilitate learners’ literacy development. Instruments and 
procedures are presented for holistic assessment of writing samples. 

What to Do 

Many useful strategies are available for helping learners acquire 
literacy. An effective approach is to begin by helping learners read 
words they already know, such as their own names, the names of their 
children, traffic signs, store marquees (e.g., K-Mart), and product logos 
(e.g., Coca-Cola). Activities b ed on environmental print can provide 
an excellent starting point for u, veloping literacy. 

For teachers who believe that instruction in phonics is an impor- 
tant part of introducing learners to literacy, phonics activities can be 
integrated easily into authentic reading and writing activities. For 
example, to help her students move from oral language to literacy. 

Dean (1992) suggests using popular songs as a starting point for 
literacy activities. She uses the Mexican song “La Bamba’ * to help 
learners associate sounds with print and to demonstrate that many 
syllables in Spanish consist of similar letter combinations: forex- 
ample, la; bam-ba; ma-ri-ne-ro. 

Another promising way to introduce reading is the Language 
Experience Approach (LEA). LEA represents both a philosophy of 
teaching and learning and a method for connecting oral language to 
literacy. It involves a staff member or another learner transcribing a 
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story that a learner or a group of learners has created based on an 
individual or shared experience. The stcries are then read aloud and 
shared with group members. After the learners become comfortable 
with the content and language, the stories can provide the basis for 
group discussions, additional reading and writing activities, or in some 
cases, language practice. Because the language used in these stories 
comes from the learners, the vocabulary is familiar, the stories are 
reflective of the learners* experience, and the sentence structures are 
comprehensible. Some literacy projects evaluate the learners’ progress 
in acquiring language by assessing the LEA stories they generate. (See 
also Taylor. 1993.) 

Assessment Strategies 

Language experience stories can be used as starting points for 
assessing reading and writing for learners at lower levels of profi- 
ciency. Other assessment strategies may include interviews that ask 
readers to talk about what they like tc read and how they derive 
meaning from print. Also helpful are surveys in which learners are 
asked to list the kinds of reading and writing they do in the course of 
their daily lives, for example, making grocery lists, checking television 
viewing schedules, reading the newspaper, observing traffic signs, and 
scanning school notices. 

As learners new to literacy make progress in the project, assess- 
ment may focus on showing how the range of literacy materials 
increases as learners become more efficient readers and w riters. At the 
end of an instructional cycle, even beginners may be able to read signs 
and ads as well as simple notes, cards, stories, and poems. By observ- 
ing reading behavior and asking learners to describe the strategies they 
use for deriving meaning, staff can document increases in 1 he learners' 
range of literacy practices. 

Learners also will increase their range of writing abilities. 

Slowly, they will write letters with more ease and require less effort to 
get words down on paper. For example, a learner may move from 
writing her name in big letters on a name card to signing a birthday 
card. As learners become more comfortable expressing their ideas, 
their writing may show a strong voice that evokes the reader’s * no- 
tion. A mother may write “My children far away. I very sad. I wish we 
together." Although the structure of these sentences is not grammati- 
cally correct, the piece has a focus, show's logical coherence, and 
clearly communicates the loss the writer feels. 

Two scoring methods can be employed for scoring learners' 
w ork: holistic and primary trait. Holistic scoring provides an overall 
view’ of the leamers'current writing ability. Scales may range from 
four point up to ten point. For example, a rating of four on a four-point 
scale could mean a more than adequate response in tei ms of organiza- 
tion. clarity, content, and mechanics; three, an adequate response; two. 
less than adequate: and one. very inadequate. 
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Because progress in 
writing is often difficult fo 
observe from day to day, staff 
may want to keep a collection 
of learners’ work in individual 
portfolios. 



1 Because reading and 
writing involve thinking 
processes that are not always 
observable, teachers need to 
be open to surprises and 
develop strategies for finding 
out How learners are thinking 
as they read and writes. 



Primary trait scoring analyzes specific dimensions, or traits, such 
as voice, sentence structure, conventions (e.g., capitalization and 
spelling), and grammar. For any one writing assignment, several traits 
or just a small number can be measured. Each trait can be scored on a 
four- or six-point range. The learners’ score can be based on individual 
traits or on the sum of the scores for all of the traits. Example 10 (see 
page 92) is an example of a primary trait scoring rubric. 

Regardless of whether holistic or primary trait scoring is used, 
staff members need to be trained in the scoring procedure. Training 
should involve staff members working together in a group to score 
some writing samples and then score samples individually. Individual 
staff members’ scores should agree before they start scoring their own 
learners’ work. Periodic checks should be made to ensure that staff 
members continue to agree on what constitutes a two or a three, for 
example, and on what scores should be given to specific writing 
samples. 

Example 1 0 is an assessment instrument de igned to evaluate 
writing samples according to the dimensions of ( 1 ) authenticity, voice, 
and engagement of the reader; (2) focus, organization, and develop- 
ment; and (3) sentence mechanics and language. The example was 
adapted from an instrument that appears in Soifer et al. (1990). Chapter 
V presents strategies for collecting, analyzing, and reporting data 
collected with Example 10. 

Because progress in writing is often difficult to observe from day 
to day, staff may want to keep a collection of learners’ work in indi- 
vidual portfolios. Writing samples for the portfolio may be chosen and 
evaluated at key points during the instructional cycle. At specified 
intervals (for example, once every' three weeks), staff members might 
identify several examples of learners’ writing that were produced 
during that time period. Staff could share their impressions as they 
review the writing samples. Staff members might invite learners to 
one-on-one conferences in which learners select what they regard 
their best or favorite piece of writing and talk about the progress they 
have made in the project. 

Assessing the progress of new readers and writers . Assessing 
the progress of beginning readers and writers can be a tricky undertak- 
ing. Three things need to be kept in mind: First, because reading and 
writing involve thinking processes that are not always observable, 
teachers need to be open to surprises and develop strategies for finding 
out how learners are thinking as they read and write. Second, reading 
and writing development are not linear processes; rather, literacy 
develops in fits and spurts. As a result, a learner might appear to be 
fairly high on a reading and writing scale one week and lower the next. 
Third, learning how lo read and write is a complex process that re- 
quires special skills and strategies — for example, being able to connecL 
meaning to print and understanding that writing represents speech. 



Chapter IV 




84 



in assessing the 
progress of beginning literacy 
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staff to be aware of the 
cyclical and recursive nature 
of literacy development. 




Literacy also involves subskills such as understanding the 
relationship between letters and sounds (phonics) and forming the 
letters of the alphabet. Another important literacy skill is sometimes 
referred to as making meaning — that is, the ability to make sense of 
written words or to express thoughts in writing. The check list shown 
in Example 1 1 (see pages 93-94) is designed to assess the meaning 
making strategies associated with literacy. 

In assessing the progress of beginning literacy students, it is 
important for staff to be aware of the cyclical and recursive nature of 
literacy development. Staff members should not be surprised if a new 
reader understands quite a bit of one story but becomes confused or 
stymied by another. Similarly, a witter may feel inspired by a particu- 
lar topic, yet have nothing to say on a different theme. 

Understanding some of the factors that help or hinder compre- 
hension and writing proficiency helps staff assess literacy progress. 

The learners' background knowledge, goals, and interests influence 
how well they are able to deal with a particular text. Thus, staff need to 
observe and listen to the learners to get a sense of the kinds of topics 
they may want to read and write about. Similarly, staff should provide 
a wide range of literacy activities (for example, listening to songs, 
labeling photographs, illustrating a story, or reading about others in the 
class) so they can respond effectively to the participants’ different 
learning strategics. Staff also should look at many different examples 
of what learners are able to read and write to get a complete picture of 
the learners’ literacy development. 

The check list shown in Example 1 1 provides descriptions of 
some of the strengths that learners exhibit as they go about making 
meaning in reading and writing. It illustrates some of the learners’ 
characteristics as they move from being relatively unfamiliar with 
literacy to becoming independent readers and writers. Not all learners 
will exhibit all the descriptors in the instrument. As staff members read 
and write with their learners, they should check the descriptors that 
characterize a particular learner. They can then compare the check lists 
of several learners to determine how the learners as a group are devel- 
oping from emerging to independent literacy. Staff members also can 
compare the check lists of several learners and summarize the data at 
the end of an instructional cycle, 

Self Assessment 

Learners are often the best judges of their own progress. Self 
assessments allow adult learners to discover what they know and do 
not know and how their competence changes over time. If connected to 
tasks, role plays, or assignments, such assessments enable learners to 
discover what is easy for them and what is difficult, allowing them to 
identify their learning needs as they think about proficiency. Self 
assessments can take various forms, including competency checklists 
where learners indicate what they can or cannot do, learning logs with 
room for open-ended responses that capture how successful a learner 
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One of the most 
successful ways of capturing 
self-assessment data in a 
systematic form is “can-do 
lists” connected to a rating 
scale. 



Effective assessment 
approaches need to take into 
ac ount the learners’ needs 
and goals and develop ways of 
showing them that they are 
making progross. 



dealt with a task, comments on portfolios, or dialogue journals between 
teachers and students. Self assessments can be created to assess each of 
the activities discussed in this chapter. 

One of the most successful ways of capturing seif-assessment 
data in a systematic form is “can-do lists” connected to a rating scale 
(see Example 12, page 95). These lists enable learners to think about 
what they can or cannot do in the context of their daily lives and allow 
them to indicate the degree to which they are experiencing difficulties 
with a task. While Example 12 addresses basic communication and 
literacy skills, similar lists can be designed to assess proficiency in 
other skill areas. The following are examples of skill areas and tasks: 

• listening comprehension — understanding teachers or classmates 
from other countries, making sense of simple recorded messages, 
understanding sales people on the telephone; 

• learning how to learn — using resources, asking for help and helping 
others, using reading and writing strategies; 

• understanding various types of texts — medicine labels, TV program 
listings, report cards, children’s books, fables; and 

• working as part of a team — setting up a time line, managing tasks, 
creating illustrations, writing texts, preparing a presentation. 

Similarly, can-do scales can be adapted to assess how much a 
person already knows about topics to be covered in a curriculum 
(positive child discipline, how the schools work, immunizations, etc.) 
A promising new area is can-do lists for technology, which allow 
learners to check what they can do and how much help they need in 
basic computer literacy. Can-do items might include turning a com- 
puter on and off, using a mouse, finding and opening a program, typing 
a few sentences, saving a document, printing, et 

If administered several times during the teaching cycle (every 
three months or so), can-do lists act as progress indicators. They 
demonstrate not only that learners are acquiring new skills but also that 
they are improving in the skills that already have been mtroduced. 

Such lists can easily be adapted to reflect the goals that learners want 
achieve or the skills, stiategies, and knowledge that the program seeks 
to impart. 

To increase their reliability as evidence of program effective- 
ness, can-do lists and scales can be combined with other measures 
that gauge the same skills, such as structured interviews, role plays, 
and various reading and writing tasks. Teachers and classmates can 
observe and give comments regarding skill levels that a learner 
exhibits on these tasks. 



Summary 

Adult learners have their own reasons for coming to class, onlv 
some of which have to do with acquiring English and developing 
literacy skills. Effective assessment approaches need to take into 
account the learners' needs and goals and develop ways of showing 
them that they are making progress. 
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Alternative assessment approaches enable staff to create project- 
based instruments and involve learners in developing the curriculum 
and designing the evaluation. The instruments may include observa- 
tion measures, protocols for interviews and focus groups, learner 
profiles, and performance samples. Staff will want to use and field test 
these instruments to measure progress reliably over time. The data 
gathered with the instruments can then be aggregated into summary 
formats and, where appropriate, quantised so that the information can 
be used to answer a variety of questions about the project. 

Staff members who use alternative assessments that are project 
based and learner focused have access to more valid and useful infor- 
mation than that which can be obtained from most standardized tests. 
To improve the use of alternative approaches to assessment and 
evaluation, project staff face the challenge of defining their vision of 
success and developing instruments that capture the changes that take 
place as language-minority adults and their children become involved 
in family literacy. 
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Example 0 

Learner Profile/Observation Card 



Type of assessment: Observation measure. 

Purpose: To record information about the knowledge, skills, and strategies that learners possess and 
the level of confidence they display in interacting with each other and to collect anecdotal evidence 
that shows ihe extent to which participants are using the information gained in class in their lives 
outside the classroom. 

Method: Open-ended charts allow teachers to take notes on what learners do and say. 



Learner Profile/Observation Card 




Learner's name 



• Communication skills 

listening and reading 
si caking and writing 



Coping strategies and problem solving 

linguistic 

nonverbal/social 



Content knowledge 

institutions 
services 
life skills 

Levels of confidence 
self 

one-on-one 



small group 
large group 











ample 



Questionnaire 



Type of assessment: Self-assessment; questionnaire. 

Purpose: To identify literacy practices in the home and record parent-child interactions. Information 
provides baseline data for documenting changes over time. 

Method: A list of activities is generated as part of whole-group or small-group discussions or after 
learners interview each other. If learners are not proficient in English, the activity can take place 
in their native language. In cases where the learners are not literate, the teacher or a literate 
partner can record responses. 

Prompt to learners: Parents and children can do many tilings together. They go to the park on 

Sunday, go fishing, cook food, clean the house, go hiking, watch TV, work in the garden, or look 
at magazines. In many families, parents help their children with homework or check their assign- 
ments. What do you do together with members of your family? 



Chapter IV 






Example 



Seif-Assessment Instrument 



Type of assessment: Self-assessment of literacy activities by frequency count. 

Purpose: To identify family activities that learners engage in at home. The instrument can be used for 
generating baseline data and documenting changes during the implementation of the project. 

Prompt to learners: You do many things together as a family. Tell us how often you do these activities. 
(Be sure to list any activities you do that are not on the list.) 



Family activity 



Go to the park 



Watch TV together 



Cook or clean house together 



Fix things that are broken 



Go fishing 



Do homework together 



Look at a book or magazine 
Play games 
Work in the garden 



Go hiking 



Almost 
every day 



Once or 



Several 



Once or 



twice a v/eek times a month ! twice a month I Seldom 




Attend cultural events ; 


, 1 




Other activities \ ' j 

i i \ 

’ i ! * 

j ; 1 ; 




! 

i 


i ; ‘ 

! i i 
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Self-Assessment Instrument 



lyp of assessment: Self-assessment; frequency count. 

Purpose: To set personal goals and develop an individualized plan for documenting progress. 

Prompt to learners: Based on the home literacy activities identified in the questionnaire (Example 4), list 
activities that you plan to undertake with your children and indicate how often you might do each. 
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oimasre 



Type of assessment: Self-assessment; open-ended questionnaire. 

Purpose: To record literacy events and learners’ accomplishments related to literacy and to compare 
goals in the learners’ individualized plans with actual activities. 

Method: Learners talk about the kinds of activities they did together with their children and discuss the 
literacy events that have taken place. Each learner or a literate partner records responses. At key 
points in the cycle the list is compared with the learner's individualized plan. The plan is then 
discussed anew and goals are renegotiated based on the learner’s current perspectives. 



Learner's name 



This week, l helped my son with his math . 
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Observation Measure for 
Listening and Reading 
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Observation Measure for 
Speaking and Writing 



Type of assessment: Observation measure. 

Purpose: To assess ' peaking and writing skills. 

Method: As learners talk about or write down instructions, staff members assess their performance and 
check the appropriate box in the chart. 

For example: After learning how to make buttered toast, learners are asked to follow oral or written 
instructions for making toast with peanut butter and jelly (familiar instructions) and bagels and cream 
cheese (new instructions). 

Learners who can produce and v- rite instructions without asking questions or looking terribly puzzled 
are rated “W ith ease." Those who occasionally ask to have a word provided, a sentence modeled, or 
seek help from peers are rated “With some support / 1 Learners who need to consult a dictionary, ask 
peers for translation, or constantly observe others are checked “With great difficulty.” 



— 






With some 


With great 


Not at 




Productive skills 


With ease 


support 


difficulty 


all 


Speaking 


Produces familiar spoken 
instructions that are 
clear and comprehensible 












Produces new spoken 
instructions that are clear 
and comprehensible 










Writing 


Produces familiar written 
instructions that are clear 
and comprehensible 












Produces new written 
instructions that are clear 
and comprehensible 
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Observation Measure for 
Nonlinguistic Domains 

Type of assessment: Observation measure. 

Purpose: To assess learners 1 performance in nonlinguistic domains related to 

learning and literacy development. The instrument can be used for generating 
baseline data to document changes during the implementation of the project. 

Method: Staff members write comments in the appropriate columns as they listen to 
learners talk about their abilities and work together in pairs or in small groups. 
Anecdotes about learners’ lives outside the classroom are also used as sources of 
assessment data. 

The following are detailed descriptors for the domain Confidence that appears 
in the observation measure. Staff can develop their own descriptors based on the 
specific objectives of their project. Numerical values can be attached to the 
descriptors to facilitate data analysis (see Chapter V for strategies for analyzing 
data). 

Descriptors for Confidence 
Level 1 

Appears shy and unsure of own abilities; does not volunteer information; 
reluctant to answer if called on; seems overwhelmed by most new tasks; needs a 
lot of support. 

Level 2 

Acts confidently when handling familiar information and seems willing to take 
risks; freely talks about himself/herself and shares information with the class; 
needs additional challenges to g r o\v and learn. 

Level 3 

Extremely confident; has a tendency to take over the class and sometimes show' 
off, causing slight resentment in the other students; requires challenging tasks 
and needs to be in groups where he/she cannot dominate easily. 

Similar descriptors can be developed for the other two nonlinguistic domains in the 
measure. Participation and Transfer to real life. 
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Example 9: Observation Measure for Nonlinguistic Domains (continued) 



Date 


Teacher 


Unit 


Class 




Confidence 



Descriptors: 

• Appears shy and unsure of 
own abilities 

or 

• Acts confidently when on 
familiar ground and seems 
willing to take risks 

or 

• Extremely confident; has a 
tendency to take over the 
class 



Participation 



Descriptors: 

• Appears overly dependent 
on one or two friends 

or 

• Uses the group as a 
resource and acts as a 
resource 

or 

• Likes to work indepen- 
dently most of the time; 
does ne t enjoy group work 



Transfer to real life 



Descriptors: 

• Does not yet use literacy 
skills outside the class 

or 

• Practices at home 



Brings examples and 
questions from home to 
the class 












Example 0 

Writing Assessment instrument 



A. Authenticity/voice/engagement of the reader 


4 = Expression strongly reflects the writer's emo- 
tional and/or intellectual involvement in the topic. 
Strongly engages the attention of the reader. 


Score: 

Comments: 


3 =The writer is engaged in the topic and engages the 
reader. 


2 = Uninteresting; not engaging; perfunctory. 


1 = Writing seems to be a mechanical exercise. 
Marked by cliches, hazy generalizations, mean- 
ingless expressions. 


. i. 

B. Focus/organizatioa/development 

Score: 

Comments: 


4 = Focuses on one main idea. Has clear beginning, 
middle, and end. Well organized and well 
developed through examples. 


3 = Focused and organized but may have a flaw in 
coherence or incomplete closure. Incomplete 
development. Explanation is strongly implicit. 


2 = Lack of clear focus, organization, or develop- 
ment. Narrative, but no explanation. 


1 = Disorganized; underdeveloped; unconnected 
generalizations. 


C. Sentence meehanics/Ianguage 

Score: 

Comments: 


4 = Few mechanical, usage, or sentence errors. 
Language used with fluency and variety. 


3 = Some minor mechanical usage or sentence errors. 
Language used competently to express ideas. 


2 = Enough usage errors to attract attention away 
from the content. Sentences understandable, but 
unconventional. 


1 = Language and mechanical errors impair meaning. 


Reprinted by permission of the publisher from Soifer, Rena et ah. The Complete Theory-to-Practicc Handbook of 



Aduit Literacy 01990 by Teachers College. Columbia University. All rights reserved, (pp. 1 78-179) 
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Checklist for Assessing Beginning 
Literacy and Emerging Literacy 

Learner’s name: Date: 

Part I: Assessing Reading 

A. As literacy starts to emerge, the new reader: 

L Z Is aware that print carries meaning but only in a very vague sense. 

2. Z Can tell print from nonprint (e.g., decoration^ but insists that she/he cannot read 

anything. 

3. I] Can tell the shape and form of English writing from other types of writing but resists 

making a guess about the meaning of a word. 

4. Z Recognizes her/his own name and a few sight words but resists predicting what a new' 

word might mean. 

5. Z Can interpret print that is part of the environment, such as reading a stop sign, 

recognizing a Coca-Cola can, pointing out the McDonald's logo, or interpreting a No 
Smoking sign, as long as a great deal of support is given. 

B. As literacy continues to develop, the new reader: 

1. Z Is able to read familiar signs, labels, and logos as long as they appear in their natural 

form (e.g., “rice” printed on bag of rice); is willing to experiment and venture some 
guesses. 

2. Zj Recognizes familiar words when written on paper, as long as some context is provided 

(e.g., a picture of a bag of rice with “rice” written next to it). 

3. Z Can read short familiar texts, such as simple language experience stories, along with 

other group members. 

4. Z Employs reading strategies that allow' her/him to move through familiar texts somewhat 

independently. 

5. Uses reading strategies that allow her/him to derive some meaning from new texts, that 
is, is beginning to read independently. 

Part II: Assessing Writing 

A. As literacy emerges, the new writer: 

1. r “ Is not yet aware of the functions and uses of writing. 

2. Z Is aware that writing carries meaning but is not yet ready to try to write. 

3. Z Copies letters and words but is not able to read what she/he writes. 
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Example 11 : Checklist for Assessing Beginning Literacy and Emerging Literacy (continued) 



4. Tries to write something, even if only a few scribbles; talks about what she has 
written; understands that writing is used to express ideas. 

5. f.J Writes her own name independently, as well as some other key words; however, 

writing still seems to be a mechanical exercise. 

B. As literacy continues to develop, the new writer: 

1 . □ Writes or copies words and is interested in what the words mean; tries out some of 

her/his own ideas as long as some support is given; may copy a word and use it to 
label a picture or photograph. 

2. □ Writes a few words or sentences independently without copying; writing may be 

uneven and meaning not always clear. 

3. □ Experiments with writing on her/his own; may write a few words about herself/ 

himself or may write about what she/he sees in a photograph. 

4. □ Experiments further with expressing ideas; writes a few words in dialogue journals 

or attempts to write parts of a language experience story; may need a great deal of 
support. 

5. □ Writes notes in dialogue journals and writes some parts of language experience 

stories independently; writing shows beginning elements of organization. 

6. □ Connects ideas and thoughts in writing; may write short description of a picture or 

an event; writes short description of seif or of family; spelling and sentence 
structure still very uneven. 
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CHAPTER V 

COLLECTING, ANALYZING, 
AND REPORTING 
ALTERNATIVE 
ASSESSMENT 
RESULTS 




Sal Gefardi 

Educational System Planning, Inc. 

Woodland, California 

This chapter deals with collecting, analyzing, using, and reporting data 
obtained with four alternative approaches to assessment and evaluation: sur- 
veys,* performance samples, interviews, and observation measures . A vari- 
ety of different analyses, techniques, and procedures are presented that can be 
used with each of the alternative approaches. However, the strategies pre- 
sented for the four instruments also may be used with other types of alterna- 
tive assessments such as portfolios and learner investigations. This chapter 
illustrates how alternative assessment data can be analyzed, interpreted, and 
reported based oil the characteristics of family literacy projects. 

The chapter is divided into four sections, each of which addresses a different 
alternative approach. Each section deals with the four key aspects of manag- 
ing the data obtained from the alternative approaches: (1) collecting the data, 
(2) analyzing the data, (3) using the data, and (4) reporting the data and find- 



* See the Glossary on pajiCN 1 1 3 i of this publication for definitions of italicized ke\ terms that 
appear in this chapter 






One of the important 
advantages of alternative 
assessment approaches is that 
they offer staff the opportunity 
to involve learners in the 
assessment process. 



ings. Strategies are presented for a variety of stakeholders such as 
project staff members, learners, advisory groups, coordinating agen- 
cies, and representatives of the funding agency. Additional informa- 
tion about alternative approaches to assessment and evaluation is 
available from such resources as Qualitative Evaluation Methods 
(Patton, 1980) and A Practical Guide to Alternative Assessment 
(Herman, Aschbacher, & Winters, 1992). 

Data obtained through alternative assessment approaches arc 
useful in the preplanning, planning, and evaluation phases of family 
literacy programs. Alternative assessment instillments are particularly 
useful in giving staff information necessary for (1) planning a program 
based on the needs of adult learners, (2) aligning curriculum according 
to the results of learner assessments, (3) monitoring learners’ progress 
toward attaining project objectives, and (4) involving learners in 
determining their progress toward mutually agreed upon goals. (See 
Chapters I and II for a discussion of objectives.) 

One of the important advantages of alternative assessment 
approaches is that they offer staff the opportunity to involve learners in 
the assessment process. Learners may be involved in such tasks as 
identifying topics or skills that will be assessed, monitoring their own 
progress toward attaining project goals, or collaborating to determine 
their rating on a performance sample. As learners collaborate in the 
assessment process, they take more responsibility for their own 
learning and for the project in general. 



Surveys 

Surveys may be used to collect information for the preplanning, 
planning, and evaluation phases of the planning process model pre- 
sented in Chapter II. In the preplanning phase, survey data are used to 
identify the needs of potential participants and to design objectives for 
the project. During the planning phase, survey data may be used to 
develop curriculum and instructional activities. This section addresses 
the use of survey results in the evaluation phase of the project. The 
information presented is based on the needs assessment survey pre- 
sented in Chapter III, Example 1 (reproduced on the following page). 

Collecting Survey Data for Evaluation 

The responses to each question on the survey should be tabulated 
to determine the range of responses and the number of respondents 
who selected each response. For example, the first question on the 
survey asks the respondents to indicate the number of adults who live 
in their apartments. The range of responses for this question would 
consist of the smallest and largest number of adults living in one 
apartment. A frequency count would indicate how many respondents 
selected each response. Data should be tabulated similarly for each 
item in the survey. The frequency and range of responses should be 
Ldlied for each question according to the type of response choices (for 
example, yes/nc, names of schools, or morning/afternoon). After the 



Assessing Success 



104 




Example© 




NEEDS ASSESSMENT SURVEY 




Name THcpIionc < ) 


Address 




Native Umgu.ipc 




l 1 low many adults In c m your .ipartmcni'' 


1 


2 How mull) children in your apartment attend Lincoln HnMicd School DistiiiT 




(Identif y individual schools by name) 




3. How many adults cuiTcntly intend GAIN program Lnglish classes ' 


3 


4 Where do they attend classes'* <c £ . Delia College, adult school) 


4 . . 


5 ll there are ndulis living m tin* apartment ulio are not attending, GAIN Knghsli 




classes, would they attend GAIN Hughs!) classes if they were held m your 




apartment complex’’ 


5 


b How many men would go ’ 


0 


7 How many women would go' 1 


7 


8 Would these adults agree to attend class tour hours a day . Inc days a week ’ 


X 


d Would tlicv need to have someone outside tlw lanuly take care ol their 




children during class - ' 


n 


Id How many children need child can* ’ 


h* 


1 1 Should class he held m the morning 01 m (he afternoon'' 


ii 


12 Is it a good idea fur us to try to gel our classes approved tor ( IAIN, or should 




we liasc our own literacy piogram ’ 


i: 


13 ll wc camioi gci our Lugli&h classes approved lo; GAIN, would you alieiul 




anyway 1 ’ 


*3 


14 1 low many hours per day should classes be held'’ 


U 


1 5 1 low many day » per week should classes Ik held ' 


15 


16 \Vli,u would you like to learn - ’ 


16 . 


a Hngiish d How the schools o|x?i ate 




h 1 lov. to gel a joh e How 10 become a cut/en 




c 1 low to Ik a potxj paiciu 




17 What is youi level of English language skills’ 




a Oral skills (l|l.ow (2) Medium (3) High 


17,i 


ii Reading and writing skills (1 1 Low (2) Medium (3) High 


1/h 


18 How long ha\c you been in the l lined States'* 


IS 


16 |low many elulilieii m your .ipjitineiii aic p.uMcip, King in i .Sl A 1 ulc \ II piupi jins ' 


r> 


Family English Literacy Project • Lincoln Unified School District * Stockton, Caltlornla 

j 



Staff may find it helpful 



to organize the data analysis 
by posing questions that focus 
on key aspects of the project. 



data have been compiled, the total frequency count should equal the 
number of people who responded to the survey. 

Analyzing Survey Data 

The results of the needs assessment can be analyzed to answer 
questions related to planning or evaluating the project. The process of 
analysis consists of reviewing the compiled data and making judg- 
ments about the needs of the learners or about how the project can e 
improved. Staff may find it helpful to organize the data analysis by 
posing questions that focus on key aspects of the project. Figure 1 
contains sample questions that staff members might use for organizing, 
analyzing, and using data that are collected with a survey. The corre- 
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sponding questions from the needs assessment survey are identified in 
parentheses. 




Sample Questions for Data Analysis 



1 . How many adults living in this community (male and female) are 
eligible to participate in the project but aro not enrolled currently 
m English-as-a-second-language (ESL) classes? (questions 1, 
3, 6, and 7) 

2. How many adults living in this community are interested in 
attending ESL classes? (questions 6 and 7) 



3. What are current and potential participants’ needs tor child 
care? (questions 9 and 10) 



4. What are current and potential participants’ preferences 
regarding the scheduling of classes? (questions 8,11,14, and 
15) 



5. What are current and potential participants’ preferences 
regarding the content of instruction? (question 1 6) 



6. What are current and potential participants’ speaking, reading, 
and writing skills? (questions 17a and 17b) 






j 



Given the high turnover 
of participants in many family 
literacy projects, a survey such 
as the needs assessment 
survey should be conducted on 
a regular basis to ensure that 
project services are based on 
current learners’ needs. 



Staff Use of Survey Data 

Survey data may be used by staff to determine how well the 
project is meeting the learners* needs. Given the high turnover of 
participants in many family literaey projects, a survey such as the 
needs assessment survey should be conducted on a regular basis to 
ensure that project services are based on current learners' needs. For 
example, the findings from Questions 1-5 can be used to identify 
important background information about the participants. This infor- 
mation also can be used to help staff determine the effectiveness of 
their recruitment efforts in various neighborhoods. In addition, it can 
help staff evaluate the suitability of current class locations and identify 
more convenient sites such as community centers or apartment 
complexes. 

During regular project meetings, staff can consider the results of 
surveys to determine the extent to which they are addressing the 
learners* needs and to deal with other issues that might have been 
identified when the survey was conducted. Such discussions enable 
staff to improve current services and to plan future activities. Survey 
data, collected annually or more frequently, can be used to answer 
questions related to developing annual evaluation reports (for 
example, “What needs have been met?"; “What new needs have been 
identified?"; “What progress has been made in meeting project 
objectives?"). 
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The reporting of survey 
results should focus on how 
staff can improve the project. 



Reporting Survey Results 

Methods of reporting the results of surveys depend on how I ho 
data will be used. If the results arc to be used for project evaluation, 
they could be organized around the evaluation questions shown in 
Figure 1 in the preceding section. The data corresponding to each 
survey question may he ( 1 ) described in brief sentences, for example, 
'‘More than half (55%) of the respondents indicated that they need to 
have someone outside the family take care of their children while they 
attend class*’ or “Approximately 30 children will need child care” nr 
(2) illustrated in tables depleting the range of responses and numher or 
percentage of responses. Figure 2 illustrates how data from Question 1 
of the needs assessment survey may be presented in a table fora needs 
assessment or evaluation report. 

The reporting of survey results should focus on how staff can 
improve the project. The conclusions or recommendations should 
prov ide answers to questions that staff and learners have posed about 
the project. Figure 3 contains a tabic and narrative that summarise the 
responses to Question 3 in Figure 1 (“What are current and potential 
participants’ requirements for child care?”) and present findings and 
recommendations. 



Performance Samples 

Performance samples are illustrations of learners' work in 
selected tasks. As one kind of performance sample, a writing sample 
can provide valid , authentic representations of learners’ progress in 
attaining project objectives. Because they can be linked directly to 
project activities, writing samples become natural extensions of the 
instructional process. Learners can produce writing samples in 
English, their native language, or both, depending on the purpose of 




Number of Adults Living in One Apartment 



Number of 
Adults 


Number/Percentage 
Responses 
Total Responses=25 


3 


8 (32%) 


4 


11 (44%) 


5 


6 (24%) 



The survey results indicate that more than half (68%) of the 
respondents live in apartments housing four or five adults. 
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the project and their literacy ability. As learners develop examples of 
their written work throughout the year, they and the staff can identify 
writing samples that demonstrate strengths as well as areas that need 
improvement. 

This section presents strategies for collecting, analyzing, using, 
and reporting the results of alternative approaches to assessing writing 
samples. Strategies refer to the writing assessment instrument shown 
in Chapter IV, Example 10 (reproduced on the following page). 

The writing assessment instrument uses a rating scale to evaluate 
the learner’s performance. A rating scale is a technique for assessing 
an area of interest (for example, reading or writing ability or self- 
confidence) on a scale of intervals usually designated by number: 1 or 
descriptors or both. Rating scales are frequently employed to differen- 
tiate performance levels. Ratings are based on the learner’s perfor- 
mance relative to narrative explanations (descriptors) of each indi- 
vidual rating on the scale. Numerical values can be attached to the 
descriptors to facilitate data analysis. Sometimes ratings are combined 
with narrative comments and placed in a performance chart to provide 
for a more comprehensive analysis (see Example 9 in Chapter IV). The 
writing assessment instrument in Example 10 enables project staff to 




Summary of Respondents’ Child Care Needs 



Itom 


Currently Serving 


Need to bo 
Served 


Conclusions 


Number of adults 
who need child care 
to attend class 


25 


35 


Ten additional adults require child 
care services. 


Number of children of 
adult participants 


30 


45 


Child care facilities are currently 
filled to capacity. Staff need to 
locate additional space for children 
of new learners. 



The original survey identified 25 adults who needed child care services in order to attend 
classes. After four weeks, 1 0 new learners requested child care services in order to attend class. 
Because the need for child care is increasing, staff recommend that additional space be located for 
children of new learners. 
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Example 0 



Writing Assessment Instrument 



1 

A. Authenticily/soi«/engagcment of the reader 


4 = Expression strongly reflects (lie writer's emo- 
tional and/or intellectual involvement m the topic 
Strongly engages llic attention of the reader 


Score 

Comments 


3 "The writer is engaged in llic topic and engages the 
reader 




2 = Uninteresting, not engaging, perfunctory 




1 = Writing seems to be a mechanical exercise 
Marked by cliches, luizy gciieralizanoiis. mean- 
ingless expressions. 


B Focus/organizaiicn/dcidopmcm 


4 = Focuses on one main idea. Has clear beginning, 
middle, and end. Well organized and well 
developed through examples. 


Score 

Commcius- 


3 •- Focused and organized but may liavc a flaw in 
coherence or incomplete closure incomplete 
development. Explanaiion is strongly implicit 




2 = l.ack of clear focus, orgaivizaiion, or develop- 
ment. Nanatn c. bat no explanation. 




= Disorganized, underdeveloped, unconnected 
generalizations. 


C Sentence mcdumcs/Lingeage 


4 = Few mechanical, usage, or sentence errors 
Language used with fluency and variety 


S/'i'iie 

Commcius 


3 = Some minor mechanical usage or sentence errors 
Language used competently to express ideas 




1 = Enough usage errors to attract audition away 
from the content. Sentences understandable, but 
uncom cmional. 




1 = Language and mrclinmca! errors impair meaning 



Rrpnntcd b> permission of ihe publislier from Soifcr. Rena el al„ The Complete Theory -to- Practice Handbook of 
Adult Literary ©1990 by Tejchcrs College. Columbia l 'Diversity All nghls reserved <pp I78-I79i 



c# Asa natural extension 
of a learning activity, a writing 
sample serves both 
instructional and assessment 
purposes, thereby eliminating 
the time and energy required 
for separate tests. 



rate the learners’ writing skills in each of three traits, or dimensions, of 
writing: authenticity, focus, and mechanics. Each of these traits is rated 
on a four-point scale, from poorer skills (1), to strong skills (4), accord- 
ing to descriptors provided for the ratings. 

Staff members could adapt the instrument by developing descrip- 
tors that would be appropriate for the learners and curriculum in their 
project. By developing project-based descriptors and associated rating 
scales, staff can improve the validity and reliability of the instrument 
and enhance the accuracy and consistency of assessment and data 
collection. The descriptors should be arranged in logical sequence with 
clear definitions for each rating. These are complicated tasks, and staff 
may wish to seek outside assistance. 

As a natural extension of a learning activity, a writing sample 
serves both instructional and assessment purposes, thereby eliminating 
the time and energy required for separate tests. By collecting and 
assessing writing samples on a regular basis and with agreed -on rating 
scales, staff members can make judgments about learners’ growth and 
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for inclusion in the portfolios 
on a specified schedule. 



about progress learners are making toward attaining the objectives of 
the project. To use rating scales effectively, staff members need to 
have a thorough understanding of the descriptors. Staff members can 
achieve this understanding by receiving training in the rating proce- 
dure and by collaborating to design the descriptors for the ratings. 

Collecting Data from Writing Samples 

Figure 4 shows how data from learners 1 writing samples can be 
collected systematically. Data obtained through this procedure can be 
used to monitor learners 1 progress in attaining the objectives of the 
project. 

The class list helps staff record assessment results of writing 
samples on a regular basis, such as weekly, monthly, or quarterly. Staff 
can use the rating scale to judge learners' writing or ask learners to 
judge their own work. To accommodate the learners 1 language profi- 
ciency levels, staff can simplify the descriptors so learners can read 
and use the rating scale for their self-assessments. In this way, assess- 
ment becrmes an instructional tool that enables learners to recognize 
and judge effective writing. Portfolios can be used to keep a record of 
selected examples of learners' writing. Although learners may write on 
a daily basis, staff can have learners select their best work for inclusion 
in the portfolios on a specified schedule. 

Analyzing Data with Frequency Counts 

Simple frequency counts provide a useful way to analyze data 
collected with rating scales. For me-e sophisticated data analysis, staff 
should seek the assistance of a professional evaluator. The analysis of 
ratings of writing samples can be facilitated by using a class list such 
as the one shown in Figure 4. Used with the writing assessment 
instrument, the list helps staff document the ratings according to the 
four descriptors ( 1 -4) in each of the three dimens : ons of writing: 
authenticity, focus, and mechanics. The list provides space for ratings 
taken at three different times — first testing, second testing, and third 
testing. This format enables staff to monitor the progress of each 
individual as well as the total class. 

The class list contains the ratings of a partial number ( 10) of 
participants in a family literacy class. The data represent the three 
assessments conducted in October, January, and April. The numbers 
under A, B, and C represent the assigned ratings according to the 
descriptors specified in the instrument. The list can be used internally 
by the staff or provided as a table in an annual evaluation report or 
final performance report. 

The data in the class list may be analyzed further by counting 
how many learners received each rating for each aspect at each data 
collection point. The tallies are converted to percentages because the 
number of learners generally varies from one testing interval to the 
next. Staff can interpret the impact of the project by examining the 
degree of change in percentages across the three testing intervals. This 




change can be illustrated graphically and documented in a matrix such 
as the one shown in Figure 5 on page 106. 

Figure 5 provides a summary of the results of writing assessments 
conducted at three testing intervals (October, January, and April). It 
displays ratings for three writing traits: authenticity, focus, and sen- 
tence mechanics. If this summary were included in an evaluation 
report, it would be accompanied by a narrative description of the 
progress and trends toward accomplishing project goats: 

The table reflects an increase in the percentage of learners who score at 
higher levels with each assessment, suggesting progress in attaining 
project objectives. The greatest growth took place in authenticity. 

Using the Results of Assessments of Writing Samples 

Staff can use the results of writing assessments for planning 
instruction, monitoring progress, determining the degree to which 
objectives have been achieved, and refining the curriculum and 
instruction. The results can help staff members form groups of learners 
with similar needs and design appropriate instructional activities. Data 
can be analyzed to indicate the progress of individual learners as well 



Class List: Results of Assessments of Writing Samples 

Instructions: Record date of the assessment and enter the rating (1-4) under each letter: 

A = Authenticity/Voice /Engagement of the Reader; B = Focus/Organization/Development; 

C = Sentence Mechanics/Language 



Name or i.D. No. uate 



B. Lopez 



J. Garcia 



M. Matello 



K. Turner 



R. Diaz 



B. Ruiz 



A. Rodriguez 



J. Fernandez 



C. Lopez 



B. Nunez 



First Testing 



Second Testing 




Third Testing 



1/15 


1 


1/15 


2 


1/15 


2 


1/15 


2 


1/15 


2 


1/15 


2 


1/15 


2 


1/15 


3 




4/2 


2 


4/2 


4 
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w When parents 
experience success at writing 
creatively about meaningful 
topics, they can transfer their 
joy of writing to their children. 



as trends of the whole class. Assessment results also can show the 
relative effectiveness of various components of the project. 

Staff may review several writing samples by one learner to 
identify changes and trends in the learner’s writing strengths. Such 
analysis can be accomplished using formal rating scales of writing 
traits, holistic scores, or without any type of actual rating. Learners’ 
writing samples may also be shared among staff to gain insights into 
the overall effectiveness of instructional activities. When learners are 
promoted to higher levels, their writing samples can be passed on to 
their instructors to ensure that instruction is appropriate and 
uninterrupted. 

Writing assessments are instructional tools as well as vehicles for 
appraising learners’ progress. When shared with the learner, the results 
provide immediate feedback that can lead to improved writing perfor- 
mance. Staff may choose to involve the leamer in developing descrip- 
tors, selecting writings to be assessed, and ultimately evaluating their 
own writing. Such self-assessments can help learners acquire an 
awareness of the characteristics of effective writing. 

When parents appreciate the importance of writing and other 
literacy behaviors, they can become more confident in supporting their 
children’s language development. When parents experience success at 
writing creatively about meaningful topics, they can transfer their joy 
of writing to their children. They may share their writing with their 
children, encourage their children to write about their own experi- 
ences, and even write stories together. 
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Summary of Three Writing Assessments 



Authenticity 




First=Oct. (n=125) 



Second=Jan. (n=118) 2% 38% 34% 26% 14% 44% 25% 17% 43% 24% 22 



Sentence Mechanics 


1 


2 


3 


4 


80% 


18% 


2% 


0 


43% 


24% 






20% 


53% 


23% 


4% 




Assessing Success 



112 










































Summarizing Writing Sample Assessments 
for Evaluation Reports 

Methods of reporting the results of writing assessments vary 
according to the purpose of the reports. One approach may be to 
present the class list (Figure 4) with an accompanying narrative in an 
evaluation report. The report could be part of an application for a 
continuation grant or used by project staff to plan additional classes 
and services. The data summarized in Figure 5 could be included in a 
Final performance report. To provide an interpretation of the data, staff 
can develop a narrative that summarizes changes that resulted from the 
project and progress made by the learners in achieving the objectives 
of the project. 

Figure 6 illustrates how staff might report the results of three 
assessments of writing samples. The example, containing a table and 
narrative explanation, is based on the writing assessment instrument 
(Chapter IV, Example 10). 



; Figure {r \ v* " - ^ 



Learners’ Growth in Writing (Authenticity) 



Authenticity 



Testing 


1 


2 


3 


H 


First Testing (Oct.) 


66% 


22% 


12% 


0 


Second Testing (Jan.) 


m 


38% 


34% 


26% 


Third Testing (April) 


o 


17% 

L 


29% 


54% 



Sample Narrative 

The table identifies the percentage of project participants 
scoring at each level (1 , 2, 3, 4) for each of the three assessment 
intervals (October, January, April). The percentage of learners 
who made progress from low levels (1 and 2) to high levels (3 
and 4) increased substantially over the three testing intervals. 
The initial assessment in October identified 66% of learners at 
Level 1 and no learners at Level 4. By the second testing in 
January, however, only 2% of the learners scored at Level 1 and 
26% scored at Level 4. By the end of classes in April, the third 
interval testing indicated that no learners £ jred at Level 1,17% 
scored at Level 2, 29% at Level 3, and 54% at Level 4. Learners 
participated in an average of three hours per week of writing 
instruction and associated activities for a total of 72 hours during 
the 24 weeks of classes. The assessment results demonstrate 
that learners have made progress in writing content that is 
authentic and engaging. 
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Interviews 



Because interviews can 
be conducted in the learners' 
native language, participants 
may be more wiBBing and able 
to provide information to staff. 



An important element of 
the focus group is the nature 
of the communication (verbal 
and nonverbal) between the 
facilitator and the respondents 
as questions, answers, and 
comments are stated and 
clarified. 



Interviews can be used to collect detailed information for assess- 
ing and evaluating the project. An interview procedure may be used 
with individuals or small groups of people such as focus groups. This 
section will deai with the collection, analysis, staff use, and reporting 
of data obtained from focus groups. 

Interviews are particularly appropriate for family literacy 
projects because participants may not have the literacy skills necessary 
forpaper-and-pencil assessments. Furthermore, because interviews 
can be conducted in the learners’ native language, participants may be 
more willing and able to provide information to staff. 

Interviews may be used independently or together with other 
measures to generate data to evaluate the effectiveness of instructional 
programs and to identify successful components of the project. They 
can be conducted by an instructor, project coordinator, or professional 
evaluator. This section of the chapter is based on interviews conducted 
with small groups — focus groups — according to a protocol that 
focuses on specific topics. Protocol is a specified set of procedures and 
questions designed to achieve the purposes of the assessment activity. 

Collecting Interview Data Through Focus Groups 

Focus group data can be collected with a question-and-answer 
procedure in which a facilitator addresses questions to group members. 
The responses are documented in writing or recorded on audio- or 
videotape. Although the proceedings may be recorded by the facilita- 
tor, staff members may want to assign this responsibility to a separate 
individual to avoid burdening the facilitator. An important element of 
the focus group is the nature of the communication (verbal and nonver- 
bal) between the facilitator and the respondents as questions, answers, 
and comments are stated and clarified. Therefore, the facilitator should 
be sensitive to the cross-cultural and other interpersonal dimensions 
that characterize group activities. 

The protocol for a focus group should be established in advance 
and linked carefully to the goals and objectives of the activity. (See 
Figure 7 on page 1 13 for an example of a protocol.) However, the 
facilitator should not discourage responses that may deviate from the 
main purpose of the session. Such flexibility creates opportunities for 
the facilitator to assess a variety of related learner needs and outcomes, 
such as language, social interaction, confidence, and critical thinking, 
that are very difficult to assess through paper-and-pencil measures. 
These issues may be recorded in anecdotal form and analyzed later 
when the tapes are reviewed. 

Interview data may be collected at intervals throughout the year, 
on a pre/post basis or at the end of instructional units. The data can be 
analyzed and interpreted in terms of longitudinal progress, pre/post 
changes, and project implementation and accomplishments. Generally, 
focus group data are collected and reported on the entire group, 




5T1 Assessing Success 



114 




although data gathered on individuals during the session also may be 
useful. 




different individuals or groups. 



Analyzing Focus Group Data 

Analyzing focus group data can be a challenging task. As they 
review large amounts of data, staff need to decide which Findings to 
analyze in detail, which ones to mention only briefly, and which ones 
to ignore altogether. The importance attached to specific Findings is 
determined by the evaluation questions, project objectives, and the 
intended audiences (for example, school board, project staff, and 
funding source). Ultimately, decisions regarding the complexity of the 
analyses should be guided by how the results are going to be used. 

Thus, the analysis could focus primarily on addressing the \u . ..na- 
tional needs of program decision makers, yet be sufficiently flexible to 
identify issues that were not anticipated. 

The goal of analyzing focus group data, is to examine systemati- 
cally the participants’ comments in order to identify trends or paltems 
that recur among different individuals or groups (Krueger, 1988). The 
outcome of the analysis may be a report that summarizes relevant 
trends, presents conclusions about the project’s effectiveness, and 
offers suggestions for improvement. To accomplish these tasks, the 
analyst must sort through an array of statements and identify opinions, 
feelings, or perceptions that are common to several participants, even 
though they may be expressed in different words or styles. Some 
attention needs to be paid to determining the range and diversity of 
participants’ perceptions. Less emphasis is given to singular opinions 
not supported by others in the group. The use of numbers or percent- 
ages may help illustrate participants’ responses, although they are not 
necessary for describing trends or supporting conclusions drawn from 
the analysis. 

A common misconception is that the analysis of focus group data 
is a simple process of Finding a few good quotes that support the 
analyst’s point of view. To produce valid conclusions, however, the 
analysis needs to be conducted systematically according to a pre- 
scribed sequence of procedures (see the three-stage process below). 

The process should allow staff to analyze a large amount of qualitative 
data and draw practical conclusions about the effectiveness of the 
project. 

Ideally, the person who facilitates the focus group should also 
analyze the results. The results can be interpreted in part based on the 
Firsthand experiences of the facilitator who actually witnessed the 
interpersonal dynamics, body language, and affective factors that 
influenced the activity. Sometimes, however, it may not be possible for 
individuals who collected the data to be involved in the analysis. For 
example, an instructor may conduct an interview with a small group of 
learners with the assistance of a bilingual staff member. Audiotapes of 
the interview, translated into English if necessary, might be reviewed 
and summarized by a professional evaluator for an evaluation report. 
Comments by those who conducted the focus group could be added to 
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To draw valid 
Inferences from the data, the 
analyst should consider such 
factors as commonly used 
words or phrases, the 
frequency of certain 
responses, and the tone and 
intensity of the speaker’s 
voice. 



the transcript, providing the evaluator with valuable additional infor- 
mation. 

According to Krueger (1988), the process of analyzing focus 
group data can be divided into three stages: ( 1) reviewing the raw data, 
(2) creating descriptive statements, and (3) interpreting the results. 

Reviewing the raw data . At this stage, staff members analyze the 
tapes or notes taken during the focus group discussion. The raw data 
consist of the participants 1 actual statements recorded in writing or on 
audio- or videotape. Also included in the raw data are the facilitator's 
descriptions of the participants, such as their attitudes and body 
language, and the general ambience of the session. 

During this stage, the analyst should focus on becoming as 
familiar as possible with the range of data collected and then begin 
organizing participants 1 responses by categories or topics. Data may be 
organized by ( l) marking all related topics in the transcript with the 
same colored marking pen, (2) extracting comments that deal with the 
same issue and writing them on a single page, or (3) rearranging the 
text according to specific categories. 

Creating descriptive statements . The next stage in the analysis 
involves reducing and organizing the raw data by creating descriptive 
statements that summarize the participants 1 responses according to 
topics. The summary also can include illustrative quotes that are 
especially insightful or indicative of the participants 1 comments. 
Comments need not be transcribed verbatim; rather, staff may prefer to 
use alternative statements that capture the speakers 1 intended meaning. 
The analyst should begin reducing the data and preparing the descrip- 
tive summary as soon as possible after each focus group session. 

Interpreting the results . At the final stage, the analyst needs to 
enhance the descriptive statements by drawing conclusions and 
interpretations from the data. The analyst should form conclusions 
about trends or patterns based on the purpose of the focus group 
session (for example, answers to staff s evaluation questions or 
evidence of progress toward meeting project objectives). Unexpected 
outcomes of the session also should be noted. To draw valid inferences 
from the data, the analyst should consider such factors as commonly 
used words or phrases, the frequency of certain responses, and the tone 
and intensity of the speaker’s voice. It is also important to examine the 
context of participants 1 responses — that is, whether the comments 
occurred in response to a specific question or were spontaneously 
triggered by an open-ended question. 

The analyst may give more weight to comments based on some 
criteria; for example, statements based on personal experience may be 
accorded more importance than something the speaker heard someone 
else say. Other factors to consider are the consistency of particular 
statements and whether individuals change their minds based on 
others’ comments. Analyzing such factors will enhance the validity 
and reliability of the findings. 
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By probing specific 
topics, the facilitator can 
uncover causes of a problem 
and identify potential 
strategies for refining and 
improving the project. 



Reports of focus groups 
should be designed according 
to the needs of the audience, 
the purpose of the report, and 
the goals and objectives of the 
project. 



Using Focus Group Data 

Staff can use focus groups to gather information that is not easily 
obtained through writte n assessments. The interactive nature of the 
focus group process allows staff to shape the direction of the session 
and to delve into selected responses. By probing specific topics, the 
facilitatOi an uncover causes of a problem and identify potential 
strategies for refining and improving the project. Focus groups enable 
staff to explore aspects of the project from the participants’ perspec- 
tives, thereby enhancing the linkage between the learners’ needs and 
the project’s objectives. 

Focus group data can be used to identify new needs of partici- 
pants as well as clarify needs that were identified earlier through sur- 
veys or individual interviews. By learning more about participants — 
their attitudes, the cultural barriers they face, their patterns of social 
interaction, and their levels of confidence — staff can make more in- 
formed choices about curriculum content, instructional approaches, 
and the overall design of the project. 

To use focus group data effectively, staff need to identify trends 
and patterns among the group’s responses. In general, the significance 
of a given issue is determined by the frequency of comments about it. 
For example, a single comment such as “The book is too hard” would 
not carry as much weight in the data analysis as five or ten statements 
about the difficulty of the instructional materials. If several comments 
were made, the staff might want to examine how appropriate the 
materials are for the learners 1 literacy levels. 

Staff also can use focus groups to obtain information about 
literacy-related events in learners' families. For example, a prompt 
such as “Tell me how you are using the home learning kits with your 
children” may help staff determine the effects of project activities on 
the learners’ families. Comments may range from “Using the materials 
weekly and enjoy them very much” (ten respondents) to “Not using the 
booklets because we don't have any crayons” (two respondents). The 
first response helps confirm the success of the project; the second 
suggests that the staff need to help some parents obtain what is neces- 
sary to use the home learning kits. 

Using focus groups enables staff to generate more detailed 
information than is possible through some other assessment proce- 
dures. For example, parenting skills, attitudes, confidence, and other 
hard-to-assess domains can be explored through focus groups. Because 
feedback is gathered directly and spontaneously from the participants, 
results are often highly indicative of the participants' perspectives. 

Summarizing Focus Group Data in a Narrative Report 

Reports of focus group interviews should be designed according 
to the needs of the audience, the purpose of the report, and the goals 
and objectives of the project. The results of focus groups can be 
communicated orally or in writing. According to Krueger (1988), focus 
group reports can be written following three different models: 
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A focus group report 
should include a statement 
describing the purpose of the 
session, an outline of the key 
questions addressed by the 
group, and a summary of the 
way the session was set up. 



Observation measures 
can be used to collect and 
record various kinds of 
information. They may focus on 
such features as learners’ 
characteristics, group 
interactions, or language and 
literacy performance. 



1 . Raw data model that presents the question or topic that the focus group 
addressed and a list of the participants’ responses. 

2. Summary description model that summarizes the results of the focus group 
and includes selected quotations from participants. 

3. Interpretive model that provides a descriptive summary, illustrative quotes, 
and a narrative interpretation of the results. 

The raw data model is the fastest and easiest to develop and is 
most appropriate for an audience that is interested in reviewing all the 
participants’ responses. This model, however, requires that readers do 
their own analysis of the data. The descriptive and interpretive models 
are usually more effective because they provide examples of the data as 
well as detailed analyses of the findings. 

A focus group report should include a statement describing the 
purpose of the session, an outline of the key questions addressed by the 
group, and a summary of the way the session was set up (for example, 
number of people interviewed and method of selecting participants). 

The results of the report should be organized and presented according to 
the key questions that the focus group addressed. It may be helpful to 
describe aspects of the session that may limit the usefulness of the data 
or prevent conclusive statements about the project. A section for con- 
clusions and recommendations is useful for integrating the key findings 
into a concise summary and providing suggestions for modifying the 
project based on the results. An appendix to the report may include the 
focus group protocol, quotes from the participants, or the entire tran- 
script. 

Figure 7 shows how to document the purpose, protocol, and re- 
sults of a focus group session that addressed parenting workshops held 
during a family literacy project. The sample narrative might appear in 
an annual evaluation report or final performance report. 

Observation Measures 

Observation measures can be used to collect and record various 
kinds of information. They may focus on such features as learners’ 
characteristics, group interactions, or language and literacy perfor- 
mance. Observations may be used as closed-ended assessments, such as 
indicating the learner’s level of listening comprehension, or they may 
be more open-ended, as when staff members judge how confident a 
learner appears when participating in a cooperative learning group. 

This section deals with the collection, analysis, staff use, and 
reporting of data obtained with observation measures. References are 
made to the measures that appear in Chapter IV to assess listening, 
reading, speaking, and writing. A modified cohort design is presented to 
illustrate how projects can analyze assessment data in a manner consis- 
tent with evaluation requirements for comparison group data. 
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Sample Summary of Focus Group Results 

The purpose of the focus group was to determine the 
effectiveness of the five parenting workshops held on Saturdays 
and the degree to which the workshops had met the needs of 
the adult learners and the objectives of the project. Thirty-five 
learners participated in the focus group. 

Protocol for the Focus Group 

Staff asked the parents to describe the parenting workshops 
they attended. Questions included: Describe how you were 
involved in selecting the topics. Were you unable to attend any 
of the workshops? Why? Do you feel you are more confident as 
a parent because of the workshops? Why? Do you help your 
children with their homework? What did you learn that you did 
not know before? What are you doing differently as a result of 
the workshops? 

At least half of the parents attended three or more of the five 
parenting workshops. They reported that they had an 
opportunity to work with staff to plan the workshops but were 
unable to recall any specific topics they had requested. All 
agreed that the workshops helped them as parents and 
enabled them to work more confidently with their children at 
home. A majority of parents indicated that they reviewed 
homework assignments, although only a small number actually 
helped their children with reading and writing assignments. 

The parents expressed general agreement about the ill 
effects of television viewing and indicated that they had been 
successful in reducing the amount of television that their 
children watched at home. Most stated that they communicated 
with their children in their native language, whereas only two 
parents indicated that they interacted with their children in 
English on a regular basis. The staff noted that, compared with 
the focus group held earlier in the year, the parents in this 
session seemed more communicative and confident when 
discussing parenting issues. 

In summary, the parents enjoyed the workshops and 
enthusiastically volunteered information during the sessions. 
Staff noted that parents who participated in both focus group 
sessions appeared more competent and confident when 
expressing their ideas in English than those who attended one 
session. Approximately half the participants could respond in 
English, and the other half required the assistance of an 
interpreter. In general, the workshops were well attended and 
seemed to provide parents with information that made them feel 
more confident about helping their children at home. Staff 
agreed that future workshops could be more effective by 
dealing with topics that are (1 ) related to parent-child learning 
activities at home and (2) linked to the curriculum used in 
weekly family literacy classes. 
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Tho descriptors should 
provide clear distinctions 
among developmental levels 
on the rating scale and 
represent appropriate growth 
according to the project’s 
literacy framework. 



The data collected with I he observation measures can be used to 
make statements about the performance of an individual or group 
according to descriptors that are associated with each of the ratings; 
With ease, With some support, With great difficulty, and Not at all, 
The instruments are depicted as examples that staff can adapt to reflect 
the objectives of their project. Staff can modify the instruments by 
designing their own descriptors and establishing their own protocol for 
rating their learners in a consistent, reliable manner. 

The descriptors reflect staff s expectations about how learners 
will perform at various stages of development. The descriptors should 
provide clear distinctions among developmental levels on the rating 
scale and represent appropriate growth according to the project's 
literacy framework (sec Chapters II and III for a discussion of the lit- 
eracy framework). The lowest rating and its descriptor should reflect 
what the staff has determined to be the lowest level of language profi- 
ciency exhibited by new enrollees. The highest rating and its descriptor 
should indicate mastery or the highest level of proficiency the staff 
consider attainable for the learners in the project. Each item on the 
scale should be mutually exclusive and part of uneven number of se- 
lections (for example, four or six), thereby eliminating middle-ground 
options and requiring observers to make clear choices. Creating well- 
developed descriptors and training staff to use the observation mea- 
sures will enhance the reliability and validity of the instruments. 

Assessment through observation means watching learners 
perform various tasks related to the project. Such assessment can he 
done without having to set aside additional time in the project for 
testing. The observations should be based on the project's objectives 
and conducted systematically and consistently at predetermined 
time intervals. 

The following is a summary of strategies that family literacy staff 
can use for obtaining accurate and meaningful data with observation 
measures. 

1 . Establish descriptors that define different levels of performance that can 
be agreed on by similarly qualified raters who are familiar with the 
instructional design of the project. The descriptors should be (a) aligned 
with the curriculum and objectives of the project and (b) reflective of the 
lowest and highest performance levels of the learners. 

2. Create protocols and record observations at regular intervals (for example, 
monthly, quarterly, or annually), The frequency is determined by how the 
data will be used and the length of instructional units (lesson, course, or 
class session). 

3. Develop an instrument for collecting and managing the data such as the 
two observation measures shown in Examples 7 and 8. 

Methods for collecting observational data should ensure that the 
form for collecting the data is completed for each learner according to 
established intervals and that each learner’s level of performance is 
recorded for each skill. The time intervals for assessment are 
determined by the techniques that will be used to analyze the results. 
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For example, a pre/post analysis requires that data be collected at the 
beginning and end of an instructional unit or specified period of time. 
To do a longitudinal analysis, staff need to collect data at more frequent 
intervals and over longer periods of time such as a year or more. Col- 
lecting data at frequent intervals for longitudinal analysis helps staff 
identify patterns and trends in the learners’ performance. Such data are 
useful for comparing the performance of the project participants with 
individuals who have similar characteristics or who have not received 
project services. 



performance. 



Data collected through 
observation measures may be 
analyzed according to a 
modified cohort design; that 
Is, growth of project 
participants can be measured 
against a comparison group 
(cohort), thereby satisfying 
one of the requirements of 
many funding sources. 



Analyzing Observation Data with a Modified Cohort Design 

Staff can analyze data obtained with the observation measures by 
reviewing the results and determining the extent to which learners are 
making progress in attaining project objectives. Stuff may want to ana- 
lyze data separately for each language group or each English profi- 
ciency level served by the project. Based on their analysis, staff can 
modify instructional activities lo enhance learner outcomes in the fu- 
ture. Additional analysis and interpretation con be conducted by assign- 
ing numerical values to the descriptors on the rating scale. Staff can 
report how many learners received what rating at each prescribed inter 
val. If the number of learners differs for each observation, the data can 
be presented as percentages of the total number of learners observed. 
Comparisons should be made through percentages, because the number 
of learners usually varies for each year of the project and for each col- 
lection interval. 

Data collect through observation measures, writing samples, 
and other types of alternative assessments may be analyzed according 
to a modified cohort design; that is, growth of project participants can 
be measured against a comparison group (cohort), thereby satisfying 
one of the requirements of many funding sources. This design is 
adapted from research by McConnell ( 1982). (See Chapter I for a dis- 
cussion of the comparison group issue.) 

The modified cohort design provides an alternative to comparing 
the project participants’ growth with adult learners who are not part of 
the* project. If the cohort design is implemented at the beginning of the 
first year of a literacy project, staff would establish comparison group 
data by collecting data on new enrollees before instruction begins. As 
new learners enroll during the project, staff would add information 
about them to the comparison group data. At a predetermined interval, 
perhaps after completing an instructional unit, a semester, or the first 
year, staff would collect observational data and measure it against the 
comparison group data to assess the learners' progress toward meeting 
project objectives. For an extended longitudinal analysis, assessment 
data could be collected during the second or third year and measured 
against the comparison group data that had been compiled to that point. 

Some funding sources may require that part of the project evalua- 
tion be based on data collected at predetermined intervals, usually every 
twelve months. Staff can meet this requirement by assessing project 
participants' performance after twelve months of instruction and com- 
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The modified cohort 
design is based on the 
assumption that the 
comparison group data 
represent levels of 
performance that would have 
been demonstrated by 
individuals who had not 
benefited from services of the 
project. 



To provide useful 
information, observation 
measures should be (1) tied 
ticrectly to the objectives and 
instructional activities of the 
project and (2) conducted on a 
regular basis. 



paring the results with the comparison group. Data on new enrol lees 
who are added to the comparison group should be collected within two 
weeks of enrollment, after the staff has become somewhat familiar with 
their background. The data become comparison group data, collected 
whenever new learners enroll during the year, thereby accommodating 
open enrollment policies adopted by many family literacy projects. 

The modified cohort design is based on the assumption that the 
comparison group data represent levels of performance that would have 
been demonstrated by individuals who had not benefited from services 
of the project — a population frequently referred to as a “nonproject 
comparison group. 1 ’ 

After the comparison group data have been established, staff can 
compare differences between the comparison group and the project 
participants in terms of the number and percentage of individuals who 
received each rating on the observation measures. Differences between 
the two groups can be illustrated through bar or line graphs that identify 
each group according to time in the project (see Figure 8 on page 120). 
Illustrations should be accompanied by a narrative that describes the 
relationship between the learners’ performance and the services pro- 
vided by the project. 

Examples 7 and 8 of observation measures, found in Chapter IV, 
contain two performance descriptors for each of the skills of listening, 
reading, speaking, and writing. By counting and reporting the number 
(or percentage) of learners observed performing at each level during 
each observation, staff can make global statements about the group’s 
abilities in each skill. The observations may show that the learners were 
uniformly low in every skill at the beginning of instruction or that they 
exhibited great diversity in their abilities. Longitudinal or post-test data 
may show that most learners were performing at more proficient levels 
or that there was considerable variation in ability. 

The modified cohort design may show that learners who have 
participated in the project are performing at higher levels than members 
of the comparison group who have not yet benefited from program 
services, thereby validating the positive effects of the project. Longitu- 
dinal data collected over several years may reveal relationships between 
the learners’ growth in performance and length of time they were 
enrolled in the project (see a graphic illustration of si^h relationships in 
Figure 8 on page 120). 

Using the Results of Observation Measures 

To provide useful information, observation measures should be 
( 1 ) tied directly to the objectives and instructional activities of the 
project and (2) conducted on a regular basis. By linking the descriptors 
and progression of ratings to instructional priorities, staff can obtain 
valuable data for assessing learners’ ongoing progress and for improv- 
ing the instructional program. The observation measures a' so can be 
used as initial assessment tools for determining learners 1 needs. For 
example, learners who are initially observed performing tasks with 
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To make the best use of 
data collected with 
observation measures, staff 
members need to examine 
individual learners' 
performance and create 
aggregate profiles of the 
whole class. 



As with other 
alternative assessments, the 
purpose of observation 
measures should be clearly 
established before an 
instrument is developed or 
used. 



ease probably would need instruction that is different from those who 
are rated as performing them not at all. 

Staff can use data obtained with observation measures to modify 
curriculum content and instructional methodok . For instance v if the 
data indicate that many learners are performing listening tasks with 
ease but writing assignments with great difficulty, staff may want to 
devote more time and effort to writing instruction. Similarly, if several 
learners are observed performing a certain task with great difficulty 
even after it has been presented and practiced many times, staff may 
want to consider how instructional activities could be modified to help 
learners gain mastery of the task. 

To make the best use of data collected with observation mea- 
sures, staff members need to examine individual learners’ performance 
and create aggregate profiles of the whole class. These findings can be 
discussed regularly at staff meetings to determine how the project can 
be more responsive to learners’ needs. For example, learners whose 
individual data contain several ratings of not at all or with great 
difficulty may have special needs that warrant more personal atten- 
tion. If the aggregate data show ratings that are spread across many 
performance levels, staff members may want to discuss how they can 
address multilevel classes more effectively. 

Individual learners’ performance records may be used to deter- 
mine whether current instructional groupings are appropriate and how 
new groups might be formed. Significant variation in ratings may be 
due to discrepancies in the staff members’ knowledge of how to use 
the observation measures. Staff may need to clarify the protocols for 
using the measures and the definitions of the performance descriptors. 
Observation measures should be used systematically according to 
predetermined criteria that are related to the project’s objectives. 
During the observations, however, staff may identify other relevant 
information that should be documented. For example, staff may note 
that some learners are frustrated, socially isolated, or have auditory', 
speech, or visual impairments. Staff also may document special talents 
and interests shown by learners. Such findings will help staff create 
activities that are more closely linked to learners’ needs and make 
other improvements to the project. 

As with other alternative assessments, the purpose of observation 
measures should be clearly established before an instrument is devel- 
oped or used. Observation measures can be used to plan instruction 
and to place learners in appropriate groups or activities. As discussed 
in Chapter IV, observational data can provide staff with information 
about learners’ ongoing progress in attaining project objectives. 
Observation measures also can yield data for annual evaluation reports 
or the final performance report. 

Reporting the Results of Observation Measures 

Reporting observational data involves summarizing the results or 
developing interpretations of the observations. Data collected from 



m 



Assessing Success 



Open enrollment and 
uneven attendance in many 
family literacy projects often 
prevent staff from assessing 
the same learners every time 
an observation is conducted. 



observational measures can be used to make descriptive statements 
about what individual learners or groups of learners can do relative to 
project goals, objectives, and instructional services. 

The observation measures found in Chapter IV are used to rale 
learners according to their receptive (listening and reading) and produc- 
tive (speaking and writing) skills in handling new or familiar instruc- 
tions. What is new or familiar depends on the learners’ prior experi- 
ences and the topics addressed in the project. Thus, the reporting of 
observation data should include descriptions or examples of the content 
or activities to which learners were responding when the observation 
ratings were made. 

Bar graphs are especially useful for presenting summaries of 
observational data. For example-, each bar could represent the number of 
learners who performed a specific skill at a given level (that is, the 
number of learners who read new instructions with great difficulty). 
The construction of the bar graph depends on what specific information 
the staff want to emphasize. Figure 8 illustrates growth in learners’ 
ability over two years’ time to produce written instructions that are clear 
and comprehensible. Figure 8 also illustrates the findings of a modified 
cohort design in which progress of learners who have participated in the 
project for one and two years is measured against the performance of a 
comparsion group of learners who had not received project services. 

Figure 9 summarizes hypothetical data for a class of 25 partici- 
pants who were rated at quarterly intervals. The table is followed by a 
narrative explanation. The numbers and narrative address only one of 
the eight areas assessed by the two observation measures. Staff could 
develop similar illustrations for data obtained for the other seven skill 
areas. When the numbers (>i) vary greatly between groups of learners or 
years of implementation, percentages can be used instead of numbers. 

As was pointed out in Chapter I, open enrollment and uneven 
attendance in many family literacy projects often prevent staff from 
assessing the same learners every time an observation is conducted. 
Therefore, staff should explain this phenomenon in the evaluation 
report and present assessment results only for the learners who attended 
regularly. Staff also may want to include in the report some of the 
causes of attrition and the strategies that are being used to improve 
retention. 

If a project has open enrollment, a second quarterly observation 
conducted in one class may be the first observation for new enrollees 
and the second observation for experienced participants. In this case, 
data gathered on several learners should be aggregated only if each set 
of data represents approximately equal amounts of instructional time. 
Data on learners who started at different times may need to be reported 
separately, or if all learners' data are combined, the first observation 
should in fact be the first for each learner. 

Each of the eight receptive and productive skills identified on the 
two observation measures should be illustrated and reported separately. 
A table that illustrated all eight skills might confuse the reader and fail 
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Summary of Data Collected with Observation Measure (Modified Cohort Design) 



“Produces new written instructions that are clear and comprehensible" 




Sample Narrative: Evaluation results based on a modified cohort design indicate that the project had a 
positive impact on the development of literacy skills among adults. Furthermore, the data suggest that 
learners who have participated for two years show substantially greater gains than those who have been 
in the project for only one year. Staff suggested that differences in performance between the two groups 
were attributed in part to the learners’ need for a full year of participation in order to make rapid progress 
in subsequent years. 
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Summary of Ratings of Four Observations (n=25) 

Quarterly Ratings on “Produces new written instructions that are clear and comprehensible" 



Rating 


September 

First 

Observation 


December 

Second 

Observation 


March 

Third 

Observation 


June 

Fourth 

Observation 


With Ease 


0 


0 


3 


10 


With Some Support 


0 


5 


12 


8 


With Great Difficulty 


5 


5 


10 


7 


Not at All 


20 


15 


0 


0 



Sample Narrative: Figure 9 shows an increase in the percentage of high ratings from the first to the fourth I 
observation and a decrease in the less proficient ratings. For the first observation in September, none of j 

the learners was able to write new instructions that were clear and comprehensible. However, by the fourth . 

observation in June, after nine months of instruction, 1 8 of the 25 learners could write new instructions with I 
ease or with some support. ; 
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to delineate important details. Staff should include a narrative interpre- 
tation of the results with each table as shown in the examples in this 
section, or they could describe the results in narrative form and place it 
with the related goal or objective as shown in Figure 10. 
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Report of Learners' Progress in Attaining Objective 
for Receptive and Productive Language Skills 

Objective - Literacy : By the end of each year of the project, 
participants will improve their receptive (listening and reading) 
and productive (speaking and writing) skills at least one level 
using observation measures developed by the project. 

Sample Narrative 

The objective was achieved during the second year of 
the project. Based on the observation measures, learners 
experienced growth in all eight literacy skill areas. The greatest 
growth occurred in listening skills with 72% of the learners rated 
with ease for that category. Fifty-seven percent received the 
rating with ease for speaking, 29% for reading, and 16% for 
writing. The results of the observation measures are consistent 
with the comments of staff who indicated that listening and 
speaking skills were emphasized during the first two years of 
the project in response to the learners' identified needs and 
requests. 

Based on the results of data analysis according to a 
modified cohort design, the project has had positive effects on 
the learners’ literacy development. The language classes, 
which integrate literacy instruction with basic life skills, 
represent a potentially successful component of the project. 



Summary 

Alternative approaches to assessment and evaluation are espe- 
cially useful for meeting the needs of participants in a family literacy 
project. The results of alternative assessment approaches can be used 
to (1) determine the extent to which a project is attaining its goals, (2) 
ascertain changes that will improve the overall effectiveness of a 
project, and (3) identify successful components of a project. 

Staff can strengthen the use of alternative approaches by ( 1 ) 
collecting data systematically; (2) analyzing the data to provide 
meaningful interpretations of the results of assessments; (3) using the 
data to answer questions that staff, learners, and others have about the 
project; and (4) creating informative reports that summarize the results 
of the assessments. 
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By considering authentic 
representations of what 
learners and staff are doing, 
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provide clear insights into tho 
effects of the project on aduft 
learners and their children. 




Surveys, performance samples, interviews, and observation 
measures are four kinds of alternative assessments that, together with 
standardized assessment measures, can be used to create a comprehen- 
sive evaluation design for family literacy projects. To maximize their 
effectiveness, alternative assessments need to be tailored to the unique 
goals, objectives, and program design of the project. By considering 
authentic representations of what learners and staff are doing, alterna- 
tive assessments can provide clear insights into the effects of the 
project on adult learners and their children. As an integral part of 
identifying needs, assessing ongoing progress, and determining project 
effectiveness, alternative approaches play an indispensable role in 
enabling staff and learners to collaborate in designing a project that is 
responsive to the needs of language- 
minority learners, families, and communities. 
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GLOSSARY 



Alternative assessment. A flexible assess- 
ment approach that is representative of the 
curriculum, meaningful to learners, and 
indicative of the learners' acquired ability or 
knowledge. Examples are surveys, interviews, 
observation measures, and performance samples. 

Annual evaluation report. Submitted yearly to 
a funding agency such us the Office of Bilingual 
Education and Minority Languages Affairs 
(OBEMLA), U.S. Department of Education, to 
summarize the accomplishments of a project. 

Assessment. The use of instruments and 
procedures to gather data on a regular basis. 
Assessment may focus on identifying learners’ 
needs, documenting the learners 1 progress 
toward meeting their own goals, and ascertaining 
the extent to which the project objectives are 
being met. 

Baseline data. Data that are collected before 
project activities are implemented. For example, 
baseline data collected on new enrollees may be 
used for comparison with data obtained from 
learners at a later point in the project. 

Can-do lists. A type of self assessment 

instrument. Learners indicate which things on the 
list they can or cannot do. It may include a rating 
scale that allows learners to indicate the deg .e 
of difficulty experienced while completing a task 
(e.g., no problem, a little difficult, very difficult). 

Criterion-referenced assessment. See 

Standardized assessment. 

Descriptor. An explanation of an individual 
rating on a scale. For example, on a 0-5 scale for 
assessing reading, a descriptor for 0 may be 
cannot read instructions, and the descriptor for 
5 might be reads instructions with ease. 

English as a second language. English 
acquired by individuals whose native. language is 
a language other than English. 

Evaluation. The process of integrating and 
analyzing assessment data at a given point in 



time for such purposes as developing and 
refining goals and objectives, documenting the 
learners’ progress, and determining the overall 
success of the project. 

Family English Literacy Program (FELP). 

A program authorized in 1984 as part of Title 
VII of the Elementary and Secondary Education 
Act to help adult English learners achieve 
competence in English. Such programs included 
instruction on how family members could 
facilitate the educational achievement of school- 
age English learners. FELP was discontinued in 
Title VII when ESEA was reauthorized in 1994. 

Family literacy. The following four federal laws 
now govern family literacy: Reading Excellence 
Act, Elementary and Secondary Education Act, 
Head Start Act, and Workforce Investment Act. 
According to their common definition of literacy 
services, a literacy program should be of 
sufficient intensity and duration to have a 
sustainable, positive impact on a family. Such a 
program should integrate interactive services 
between parents and children, support parents 
and other caregivers as the primary teachers for 
and full partners in the education of their 
children, provide educational and related 
services that lead to economic self-sufficiency, 
and prepare children for success in school and 
life experiences. In short, family literacy focuses 
on helping family members achieve their goals. 

Final performance report. Submitted to a 
funding agency such as the Office of Bilingual 
Education and Minority Languages Affairs 
(OBEMLA), U.S. Department of Education, to 
summarize the accomplishments of a project 
during the entire grant period. 

Focus group. A group that is formed for the 
purpose of providing input. Group members are 
interviewed according to a protocol that has 
been previously established. 

Frequency count. A technique for tallying 
information, usually from survey responses. A 
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frequency count may provide information such as 
Iiuw many parents need child care or how often 
participants visit the library. 

Holistic scoring. A strategy for assessing 
performance by summarizing individual features 
to obtain an overall view of learners’ progress or 
some other dimension of the project. 

Instructional program. A coordinated set of 
activities that represent the curriculum and 
instructional components of a project. 

Interview. An alternative assessment approach for 
obtaining detailed information. An interview may 
be administered to individuals or to groups, with 
respondents answering orally or in writing. 
Interviews may be used with focus groups. 

Levels of performance. Definitions of 
acceptable performance in terms of a specific 
numeric criterion. An example of a level of 
performance would be “individual learners 
complete 90 percent of outcomes on a 
performance assessment checklist.' 1 

Literacy. The ability to use language and 
language-related strategies, especially those 
associated with reading and writing, in a variety 
of settings (family, community, school, work) to 
achieve one’s own goals (personal, professional, 
academic). 

Literacy framework. Staff members’ and 
learners’ shared understanding of the learners’ 
literacy needs, program goals, curriculum and 
meLhodology, and approaches for assessing and 
evaluating literacy development. The elements of 
the framework should be coordinated so that they 
are interdependent and mutually supportive. 

Literacy practices. The ways individuals use 
literacy in their daily lives. 

Modified cohort design. A method of 

analyzing data by comparing the performance of 
project participants with comparable individuals 
who have not yet benefited from project services. 

Norm-referenced assessment. See 

Standardized assessment. 



Objective. A description of an outcome or 
change that is expected as a result of 
implementing a project. The content of 
objectives is determined by what the learners 
need, what the staff arc prepared to leach, and 
what is possible to achieve within the time and 
budget constraints of the project. 

Observation measure. An alternative 
assessment instrument for obtaining various 
kinds of information. Observations may focus 
on such features as learners’ characteristics, 
group interactions, or language and lileiacy 
performance. 

Performance measures. Assessment tools 
used to determine the extent to which an 
indicator of progress (e.g., skill level gain in 
listening, speaking, reading, or writing; 
certificate completion; or placement in job 
training) is present or achieved. 

Performance sample. An example of a 
learner’s work in selected tasks. Performance 
samples include writing samples, oral readings, 
role plays, and oral presentations. 

Portfolio. A collection of examples of a learner's 
accomplishments such as reading logs, language 
experience stories, and writing samples. The 
contents of the portfolio can be used to show a 
learner’s progress over time. 

Project. Refers to all components to be 

implemented, including those described in the 
instructional program. 

Protocol. A specified set of procedures and 
questions designed to achieve the purposes of an 
assessment activity. 

Rating scale. A technique for assessing an area 
of interest (for example, reading ability or self- 
confidence) on a scale of intervals usually 
designated by numbers or descriptors. 

Reliability. Refers to the capacity of an 

assessment instrument or procedure to produce 
approximately the same results consistently on 
different occasions when the conditions of the 
assessment remain the same. For example, will 
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the instrument provide the same results if 
learners are tested on Thursday instead of 
Wednesday, or if two or more competent judges 
scored the test independently? 

Standardized assessment. An instrument 
created according to explicit specifications, with 
test items selected for difficulty and 
discriminating power. A norm-referenced 
assessment compares an individual’s current 
achievement to the average performance 
(norms) of selected participants (norming 
group). A criterion-referenced assessment 
compares an individual’s achievement to an 
absolute standard or criterion of performance. 

Survey. An alternative assessment approach for 
obtaining information. A survey may be 
administered to individuals or groups, with 
respondents answering orally or in writing. 

Validity. Refers to the capacity of an assessment 
instrument or procedure to measure what it 
claims to measure. For example, does the 
reading test, actually measure reading 
comprehension, or docs success on the test also 
depend on the learners’ knowledge of t IS. 
culture? 
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cooperative, 72-73 
instructional, 28, 64. 65-68 
project, 3 1 , 68 

affective factors, 74 
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appropriate uses of standardized 
tests, 5-6 

assessment, 3, 129 
closed-ended, 39 
collaborative, 19 
conventional, 64 
framework, 21 
initial, 14, 37-39 
needs (see needs assessment) 
open-ended, 39, 85 
progress, 63-64 
project-based, 21, 80 
self, 78-79, 83-87, 95 
standardized, 1-2, 15, 35, 64, 
122, 131 

strategies, 31, 67, 70, 71, 73, 
75, 76 

assessment, alternative, 1, 129 
design, 32 
instruments, 19, 31 
interviews, 23, 39, 41, 108-112 
learner-centered, 66, 80 
observation measures, 8, 23, 
75, 90-91, 112-122 
performance samples, 23, 10 1 - 
107 

portfolios, 47, 79, 104 
procedures, 31, 38 
surveys, 39-41, 98-101 

attendance patterns, 1 1 



B 

baseline data, 1 1, 27, 47, 67, 70, 
129 

Basic English Skills Test (BEST), 
4 

c 

Cambodians, 40-42 
(see also Khmer) 

can-do lists, 15, 79, 95, 129 
Chinese, 30 
citizenship, 40 
cognition, 74 
collaboration, 25 
community liaisons, 32 
comparison group, 13 
competency checklists, 78 
computers, 30 

Comprehensive Adult Student 
Assessment System (CASAS), 
4 

criterion-referenced, 4 
cultural 

background, 39 
cross-cultural sensitivity, 25, 
66-67 

cross-cultural strategies, 45 

curriculum, 64 
content, 22 
development, 28 
leamer-centcred, 65 
literacy, 22 

D 

data 

analyses, 31 

baseline, 11,27,47, 67, 70 
collection, 34 
demographic, 39, 41 



focus group, 4 1, 108- 1 12 
frequency count, 31, 99, 104 
interpretation, 1 10 
longitudinal, 117 
observation, 1 14 
raw, 110, 112 
survey, 98-101 

demographics, 39. 41 

descriptor, 63,67-68,72, 78, 129 

development 
cognitive, 22 
linguistic, 22 

dialogue journals, 79 

discrete skills, 18 

E 

Elementary and Secondary 
Education Act (ESEA), 2, 25, 
27, 40 

English as a second language, 129 

enrollment, 28-29 

evaluation, 3, 34, 129 

modified cohort, 112, 116-119 
plan, 1 
process, 22 
product, 33 

evaluator, project, 32, 34 

F 

Family English Literacy 

Programs (FELP), 21, 26-30, 
129 

family literacy, 10, 129 

final performance report, 104, 

107, 112, 118, 129 

finding an appropriate 
standardized test, 6 

focus groups, 8, 39, 41, 80, 108- 
113, 129 
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framework 

literacy, 19,21-22, 25,42 
Freire, Paulo, 32-33, 74 

frequency count, 9, 31. 98, 104, 
129 

funding sources, 42 

G 

goals (see also objectives) 
program, 2 
statements, 25 

grammar, 64 
grant proposals, 42 

H 

Hmong, 30 

i 

instruments 

project-based, 80 
self-assessment, 83-87 

intake, 14 

assessment, 45-47, 57 

integrating program planning, 
implementation, and evaluation 
activities, 14 

intelligence, 18 

interviews, 8, 23, 39, 41, 45-46. 
79, 108-113, 130 

K 

Khrner, 30, 40 
Korean, 30 

L 

language (see also native 
language) 
abilities, 71 
groups, 39 

productive skills, 1 14 
receptive skills, 1 14 

language minority 
adults, 20 

adults and children, 80 



Language Experience Approach 
(LEA), 75-76 

Lao, 30 

(see also Hmong) 

learner (see also participants) 
behavior, 68 
needs, 3, 78, 79 
outcome, 12 
profiles, 68, 80 
progress, 3, 13, 78 

learner centered 
approach, 65 
assessment, 66 
curriculum, 65 
programs, 21 

learning 
logs, 78 
strategies, 20 
views of, 18-19 

levels of performance, 1. 130 

limitations of standardized tests, 5 

literacy, 3, 130 

activities, 19, 20, 74, 78 
awareness, 27 
brokers, 66 
contexts, 20 
curriculum, 22 
definition, 21-22 
development, 21, 78 
emerging, 78 
experiences, 43 
framework, 12, 19-22, 25, 42, 
130 

inventories, 65 
paradigm, 43 

practices, 13, 20-21, 66, 70, 

130 

quantitative, 20 

longitudinal 

analysis, 1 16-1 19 
data, 117 
progress, 108 



M 

modified cohort design, 112, 116- 
119,130 

N 

National Council of La Raza 
FELP project, 26 

native language, 10, 39, 43, 45, 

46, 67, 108 
literacy, 43 
proficiency, 43 

needs assessment, 14, 26-27, 38- 
39, 42, 50-53 

norm-referenced assessment, 4, 
130 

o 

objectives. 31. 130 

instructional, 19, 27, 38 
program, 2 
project, 35, 64 

observation 
card, 69 

measures, 8, 23, 75, 90-91, 

1 12-123, 130 

p 

parent-child interactions, 10, 17, 
26-31,68-70,72-73,106 

parent survey, 44, 52-53 

Parenting Curriculum for 

Language Minority Parents, 30 

participants 
enrollment, 28 
recruitment, 28-29 

performance 
levels, 1 

measures, 1, 130 
samples, 8, 23, 80, If ’-107, 

130 

phonics, 64, 75, 78 

placement, 14 
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planning, 32-33 
collaborative, 21 

planning process model, 23-25 
portfolios, 47, 73,77, 104, 130 
proficiency levels, 12 

program 

effectiveness, 79 
goals and objectives, 2 
instructional, 1, 130 

progress 

assessment, 63-64 
indicators, 2, 64, 79 
longitudinal, 108 

project, 130 

evaluator, 32, 34 
goals, 25 
objectives, 64 

protocol, 14, 80. 108. 130 

Q 

questions 

closed-ended. 39 
open-ended, 39, 1 10 

R 

rating scale, 9, 102, 130 
reading 

proficiency, 46, 47 
strategies, 79 
tasks, 79 

recruitment 
staff, 28-29 
student, 28-29 

reliability, 4, 72, 130 

s 

samples 

performance, 23, 101-107 
writing, 14-15.47,77. 102. 
104 

scoring 

holistic, 27, 76, 130 
primary trait, 77 



skill 

areas, 79 
levels, 79 

skills 

discrete, 18 
Spanish, 30 
staff 

bilingual, 29, 39-40 
recruitment and training, 29 

stakeholders, 29, 34, 98 

standardized 

assessment, 1, 35, 64 
measures, 64 
tests, 80 

standards 

curriculum, 2 1 

stimulation 
external, 19 

surveys, 8. 39-41,98-101, 131 
results, 44, 101 

T 

tests, standardized, 43 
Basic English Skills Test 
(BEST), 4, 7 

Comprehensive Adult Student 
Assessment System 
(CASAS), 4,7 

time line, 29 



u 

United States Refugee 
Resettlement Program, 41 

v 

validity, 4, 131 
Vietnamese, 30, 40 
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Workforce Investment Act of 
1998, 1 

workplace literacy, 10 

writing 

abilities, 76 

assessment, 102-103, 105-107 
samples, 14-15 47, 77, 102, 
104 

strategies, 79 
tasks, 79 




Rrvised Edition 

Daniel D. Holt and Carol H. Van Duzer 

Editors 

Programs that teach adult English language learners now face increase 
accountability demands from their funders to meet program and learner goals. 
Assessment and evaluation play a critical role in measuring progress toward 
these goals. This updated and revised version of Holt's 1994 Assessing Success 
in Family Literacy Projects provides guidance on developing an effective 
evaluation plan for adult English language programs - whether in the context 
of family literacy, workplace and workforce literacy, or general language 
development. With an emphasis on surveys, interviews, observation measures, 
and perfonnance samples, the authors show how staff members and learners 
can gain accurate information about how well they are meeting their goals. 
The book provides many sample assessment tools and examples of strategies 
for summarizing and analyzing assessment data that can be customized. 
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